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ABSTRACT 

This module, one in a series of performance-based 
teacher education learning packages, focuses on a specific skill that 
vocational educators need to create appropriate learning environments 
and to plan and manage instruction that is well-suited to the 
learning and psychological needs of today's adults. The purpose of 
the module is to teach instructors how to evaluate learners' progress 
in meeting specified objectives. Introductory material provides 
terminal and enabling objectives, a list of resources, and general 
information. The main portion of the module includes three learning 
experiences based on the enabling objectives: (1) demonstrate 
knowledge of the rationale for and factors involved in student 
evaluation; (2) critique a test; and (3) critique a case study on 
developing evaluation plans for adult learners. Each learning 
experience presents activities with information sheets, samples, 
worksheets, checklists, and self-checks with model answers. Optional 
activities are provided. Completion of these three learning 
experiences should lead to achievement of the terminal objective 
through the fourth and final learning experience that requires (1) an 
actual teaching situation in which to evaluate the performance of 
adults, and (2) a teacher performance assessment by a resource 
person. An assessment form is included. (YLB) 
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FOREWORD 



This module Is one of a ser;es of over 130 performance-based 
teacher education (PBTE) learning packages focusing upon 
specific professional competencies of occupational instructors 
(teachers, trainers). The competencies upon which these modules 
are based were identified and vejfied through research as be- 
ing important to successful teaching. The modules are suitable 
for the preparation of instructors in all occupational areas. 

Each nriudjle provides learning expenences that integrate theory 
and application, each culminates with criterion-referenced assess- 
ment of the instructor's performance of the specified competen- 
cy. The materials are designed for use by teachers-in-training 
working individually or in groups under the direction and with the 
assistance of teacher educators or others qualified to act as 
resource persons Resource persons should be skilled in the 
teacher competencies being developed and should be thoroughly 
oriented to PBTE concepts and procedures before using these 
materials. 

The design of the materials provides considerable flexibility for 
planning and conducting performance-based trair mg programs 
for preservice and tnservice instructors, as well as business- 
industry-labor trainers, to meet a wide variety of individual needs 
and interests. The materials are intended for us& by local educa- 
tion agencies, postsecondary institutions, state departments of 
educat;on, universities and colleges, and others responsible for 
the professional development of instructors. 

The PBTE modules in Category N— Teaching Adults— are de- 
signed to enable adult instructors to create appropriate learning 
environments and to plan and manage instruction that is well 
suited to the lec.rning and psychological i ^eds of today's adults 
The modules are based upon 50 competencies identified and 
verified as unique and important to the instruction of adu'ts. 

Many individuals have contributed to the research, development, 
field review, and revision of these training materials. Apprecia- 
tion is extended to the following individuals who, as members 
of the DACUM analysis panel, assisted National Center staff in 
tb«* identification of the competency statements upon vyhich this 
category of modules is based: Doe Hentschel, State University 
of New York at Brockport; David Holmes, Consortium of the 



California State University, Joanne Jorz, JWK International Cor- 
poration, Virginia, Jean Lowe, Fairfax County Public Schools, 
Virginia, Jinn Menapace, BOC/Lansing-Generai Motorf , 
Michigan, Norma Milanovich, University of New Mexico, Cuba 
Miller, Sequoia Adult School, California, Donald Mocker, Univer 
sity of Missouri, and Michael A. Spewock, Indiana Univeroity of 
Ponnsylvania. 

.\ppreciation is also extended to the following individuals for their 
critical field reviews of the six modules in the N category during 
the development process. Edv/ard K. Allen, Donna Baumbach, 
Ronald J. Bula, Madelyn R. Callahan, Deborah Clavin, Joe 
Cooney, Yvonne Ferguson, Howard Hams, Ronald Hilton, David 
Holmes, Donna E. Johnson, Edward V. Jones, Russell Kratz, Jean 
Lowe, Frances Malanga, Donald L. Martin, Sandy McGechaen, 
Norma Milanovich, Audni Miller-Beach, Donald Mocker, Christa 
Oxford, William Reese, Rick Schau, Steven E. Sorg, Michael A. 
Spewock, Neal Wiggin, and James L. Wright. 

Recognition for major individual roles in the development of these 
matenals is extended to the following National Center staff. H^rry 
N. Drier, Associate Director, Development Division, and Robert 
E. Norton, Program Director, foi leadership and direction of the 
project, Lois G. Harrington, Proeram Acsociate. for training of 
module writers, assistance in the conceptualization and develop- 
ment of the materials, and maintenance of qualit/ control, David 
J. Kalamas, Gradui \e Research Associate, for development of 
illustration specifications; Susan Dziura, for \r.'i\a\ art work: and 
Shellie Tremaine and Cheryl Salyers, for their word processing 

Special recognition is also extended to the staff at AAVIM for their 
invaluable contnbutions to the quality of the final printed products, 
particularly to Sylvia Conine for typesetting, to Marilyn MacMillan 
for module layout, design, and final art work, and to George W. 
Smith, Jr for supervision of the module production process 
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career planning, preparation, and progression The National Center 
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• Installing educational programs and products. 
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Direction is given by a representative from each of states, 
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organizations, government agencies and Industry. 



INTRODUCTION 



Evaluation is an integral part of the teaching/learn- 
ing process. It staits when you determine the train- 
ing needs of yourstu^ ^nts (Module N-3) and develop 
instructional objectives designed to meet those train- 
ing needs (Module N-4). This is the preassessmcnt 
and planning phase of instruction. 

In this module, the focus is on evaluating learners' 
progress in meeting the specified objectives. Stu- 
dent progress needs to be evaluated frequently so 
both you and your students know what they have ac- 
complished and what still needs to be done. Student 
progress also needs to be evaluated frequently so 
students receive the positive feedback needed to 
stay motivated. 

Through postassessment— end-of-course evalu- 
ation—you can verify that all objectives have (or 
have not) been met according to specified standards. 

Preassessment gives you a starting reference 
point. You can measure "progress" because you 
know where your students were when they started. 
Performance objectives give you an ending 
reference point. You know whether students have 
accomplished what they set out to accomplish 
because their objectives have been clearly stated, 
in observable terms. 

In short, through an effective evaluation pro- 
cedure, you can answer a number of essential in- 
structional questions: What is being learned? How 
well is it being learned? How much is being 
learned? How effective are the learning 
experiences? 

Such questions need to be answered regardless 
of the ages of the students involved. Adults tend to 
be utility-oriented learners who bring specific needs, 
objectives, and goals to the learning situation. It is 



therefore important not c nly that the r needs be met, 
biit that they have tangible proof of the dividends 
received from their investment of time and energy 
in the learning program. There are a variety of 
evaluation strategies that can help provide this proof. 

The benefits of employing an effective evaluation 
procedure are at least fourfold. (1) you gam impor- 
tant information on learners' progress and can make 
appropnnte adjustments to your instructional plans, 
(2) learners obtain information about their achieve- 
ment in the program. (3) learners develop skills in 
evaluating their own work more objectively, thus 
gaining a valuable tool for future use, and (4) deci- 
sion makers secure information they need to make 
decisions about whether to continue, restructure, or 
discontinue particular programs. 

In this module, basic information on evaluation 
is provided in Learning Experience I. Information on 
testing— how to develop tests, the uses of test 
results, and grading— is addressed in Learning Ex- 
perience II. Evaluation strategies that are particularly 
appropriate for use with adult learners are dis- 
cussed in Learning Experience III. 

There is a logical reason for this particular orga- 
nization—for not discussing evaluation and testing 
in terms of adult learners throughout the module. 
New instructors need basic information about evalu- 
ation and testing, as well as information on how to 
evaluate adult learners. Experienced instructors of 
younger students, who already have skill in evaluat- 
ing student performance, need only the adult- 
specific information. Thus, the module is organized 
to accommodate, as efficiently as possible, the 
needs of both groups— to allow each to develop the 
specific skills they need without having to complete 
activities covering skills they already possess. 
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Objectives 
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l(Leai;nlnglExperienc^^^^ ; . - r ^ . s t v 
Enabling Objectives: 

1. After completing the required reading, demonstrate 
knowledge of the rationale for and major factors in- 
volved in student evaluation (Learning Experience I) 

2. After completing the required reading, critique a test 
{Learning Experience II). 

3. After completing the required reading, critique the per- 
formance of an instructor in a given case study in 
developing evaluation plans for use with adult learners 
{Learning Experience III). 

Prerequisites 

To complete this module, you must have knowledge of the 
characteristics of adult learners and the process of adult 
development. If you do rot already meet this requirement, 
meet with your resource person to determine what method 
you will use to do so. One option is to complete the infor 
mation and practice activities in the following module: 

• Prepare to Work with Adult Learners, Module N-1 

Resources 

A list of the outside resources that supplement those con- 
tained within the module follows. Check with your resource 
person (1) to determine the availability and the location 
of these resources, (2) to locate additional references in 
your occupational specialty, and (3) to get assistance in 
setting up activities with peers or observations of skilled 
teachers, if necessary. Your resour/:e person may also be 
contacted if you have any difficulty with directions or In 
assessing your progress at any time 

Learning Experience I 

Optional 

Reference: Morris, Lynn Lyons, and Fitz-Gibbon, 

Carol Taylor. How to Measure Achievement. Beverly 

Hills, CA: Sage Publications, 1978. 

Reference: Popham, W. James. Modern Educational 

Measurement. Englewood Cliffs, NJ: Prentice-Hall, 

1981. 

Reference: Gronlund, Norman E. Measurement and 
Evaluation in Teaching. Fifth Edition. New York, NY' 
Macmillan Publishing Co., 1985. 
Reference: Kirkpatrick, Donald L. How to Improve Per- 
formance Through Appraisal and Coachmg. New York, 
NY: American Management Association, 1982. 
References: The National Center for Research in 
Vocational Education. Professional Teacher Educa- 
tion Module Series; Modules D-1, D-5, and K-3. 
Athens, GA: American Association for Vocational In- 
structional Materials, 1984-86. 



Learning Experience il 

Optional 

Referc nee. Denova, Charles C. Test Construction for 
Training Evaluation. Madison, Wl. American Society 
for Training and Developfnent, 1979. 
Reference: Mager, Robert F., and Beach, Kenneth M., 
Jr. Developing Vocational Instruction. Belmont, CA: 
Pitman Learning, 1967. 

Reference. The National Center for Research in Voca- 
tional Education. Professional Teacher Education 
Module Series; Modules D-2, D-3, D-4, K-4, and K-5 
Athens, GA: American Association for Vocational In- 
structional Materials, 1983-86. 

Learning Experience III 

Optional 

Reference: Knowles, Malcom S. The Modern Practice 
of Adult Education: From Pedagogy to Andragogy. 
Revised Edition. Chicago, IL: Follett Publishing Co., 
1980. 

Reference. Knox, Alan B. Adult Development and 
Learning. A Handbook on Individual Growth and Com- 
petence in the Adult Years for Education an < the Helping 
Professions. Higher Education Series. San Francisco, 
CA: Jossey-Bass, 1977. 

Learning Experience IV 

Required 

An actual teaching situation in vyhich you can evaluate 
the performance of adults. 
A resource person to assess your competency in 
evaluating the performance of adults. 

General Information 

For information about the general organization of each 
performance-based teacher education (PBTE) module, 
general procedures tor its use, and terminology t,iat is 
common to all the modules, see About Using the National 
Center's PBTE Modules on the inside back cover. For 
more in-depth information on how to use the modules in 
teacher/trainer education programs, you may wish to refer 
to three related documents: 

The Studer}t Guide to Using Performance- Based Teacher 
Education Materials is designed to help orient preservice and 
inservice teachers and occupationat trainers to PBTE in 
general and to the PBTE materials. 

The Resource Person Guide to Using Performance-Based 
Teacher Education Matenals can help prospective resource 
persons to guide and assist preservice and tnservtce teachers 
and occupational trainers in the development of professional 
teaching competencies through use of the PBTE modules. 
It also includes lists of all the module competencies, as well 
as a listing of the supplementary resources and the addresses 
where they can be obtained. 

The Guide to the Implementation of Performance-Based 
Teacher Education is designed to help those who will admin- 
ister the PBTE program. It contains answers to implementa- 
tion questions, possible solutions to problems, and alternative 
courses of action. 
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Activity 



Effective evaluation is an essential element in adult learning. It can provide 
both you and the learners with valuable information about learners' skills, 
achievements, and instructional needs. This knowledge is essential to your 
ability to develop instructional plans that will continue to meet students' in- 
dividual needs. For information on why, when, and how to evaluate, how to 
grade, and how to use evaluation results, read the following information sheet. 



STUDENT EVALUATION 

Evaluation is a common and natural human activ- 
ity. We all do it regularly, whether we are consciously 
using rigorous and clearly defined standards or tak- 
ing a more casual and intuitive approach to making 
value judgments about daily situations. 

Regardless of how we go about it, the process of 
evaluation acts as both a scale and a compass. It 
acts as a scale in that we measure the value of our 
gains and losses. It acts as a compass in the sense 
that Wo take a directional reading to determine where 
we stand in relation to the things that we value most. 

If the evaluation process is accurate and we use 
it properly, we gain valuable information that can 
help guide our course of action. And if that course 
of action leads to meeting our objectives, it can 
renew our confidence in our ability to succeed and 
encourage us to continue our efforts. 

If the piocess is inaccurate or inappropriately 
used, the negative effects are also f'^^lt. Theee effects 
are, at their best, demoralizing; and at their worst, 
loaded with damaging consequences 

Effective evaluation thus depends not only on 
developing accurate assessment procedures, but on 
acquiring the skills necessary for using them well 
You need to devise good data-collection techniques, 
which will yield good data, and you need to be able 
to make sound judgments in analyzing and using 
those data for instructional purposes. 




Why Evaluate? 

Many adults come to a learning situation with 
specific needs that are critical to their professional 
or personal success. Because the time and energy 
that they can invest is often limited, they want tan- 
gible evidence that thetr investment is well-placed. 
Information that provides learners with a view of their 
strengths, achievements, and the direction in which 
their work is heading gives them that evidence. 

Learners need this evidence not only in formal 
learning situations, but in indeoendent learning activi- 
ties as well. Expenence with a variety of assessment 
techniques can enable learners to develop effective 
and realistic approaches to assessment that they 
can use on their own. 

The benefits of evaluation extend well beyond pro- 
viding learners with this type of evidence. When 
students of any age obtain immediate, frequent, and 
positive feedback about their progress, they con- 
sistently learn more, retain more, and are more 
motivated to continue with their learning activities. 

As an instructor, you also will need the informa- 
tion that Co. I be obtained from assessment activities. 
It can help you identify learners who are ready to 
move on to new learning activities and those who 
need additional help in certain areas. It can also help 
you determine what changes you need to make m 
your instructional plans in order to meet individual 
needs. 

Bear in mind that you, too, need the type of 
positive reinforcement that learners gain from know- 
ing what they've achieved. This comes from seeing 
the evidence thc.t students are acquiring needed 
skills and knowledge and from knowing that you are 
providing effective instruction to help make it 
happen. 

A final reason to evaluate is that your own 
employer is likely to require that you evaluate and 
report on learners' progress at specified times. 
Education is expensive, and cost factors must be 
considered. Businesses and industries operating 
training programs are interested in the "bottom 
line". Is their investment paying off? Is the training 
program cost-effective? Are trainees learning the 
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skills they need in aA efficient manner? Schools and 
colleges, too, are operating more and more as 
businesses, with accountability concerns and a 
cost/benefit focus. 

Student performance thus may have a great im- 
pact on the fate of a given program. If students in 
a particular program are not meeting the specific ob- 
jectives, then the usefulness of that program is in 
question. Does the program need to be modified 
significantly? Should it be eliminated? Is the problem 
not in the program, but in the instruction? In short, 
program evaluation and teacher evaluation are 
directly related to student learnmg and success. 

Evaluation Terminology 

Because evaluators use some terms in very 
specialized ways, let's clarify our use of terminology 
before going any further. 

Evaluation. Evaluation is a process by which you 
(1) gather information about the quality of the 
teaching/learning process and the achievement of 
objectives and (2) use that information to make 
educational decisions and improve the instructional 
process. 

Assessment. Assessment is the use of specific 
techniques to gather that information. Thus, 
theoretically, you can assess (gather data) without 
evaluating— without analyzing and making use of 
those data. When a student gets an F on a unit test 
and then goes on to the next unit without having 
mastered the previous unit, that's assessment 
without evaluation. 

Ideally, however, assessment results should 
always be used not only to k entify the amount of 
student progress and level of student achievement, 
but also to remedy any problems identified, 't is by 
using assessment results that you can best serve 
those students presently enrolled in your program, 
as well as future students. 

Testing. Testing is not a synonym for assessment; 
it is one type of assessment. The evaluation process 
can and should include a variety of assessment tech- 
niques: standardized tests and teacher-made tests; 
objective tests and subjective tests; tests of knowl- 
edge, skilb, and attitudes, interviews and discussions, 
observations, and self-reports—jusi to mention a 
few. The assessment techniques can be formal (e.g., 
a structured test) or informal (e.g., a chat with a stu- 
dent who appears to be having trouble completing 
a lab assignment). 



Performance. When training and development 
people use the term performance, they generally 
mean "behavior on the job." Thus, evaluation of per- 
formance would mean evaluation of on-the-job perfor- 
mance. This is not how we are using the term in ♦^is 
module. 

When we say in the title of the module. Evaluate 
the Performance of Adults, we are using the term per- 
formance as it IS used in everyday English. It means 
the evaluation of learners' accomplishments in the 
program. What do they know? What attitudes do 
they possess? What skills have they mastered? In 
performing skills, do the products they produce 
andfor the processes they employ meet established 
criteria? 

Depending on the instructional setting, skills may 
be assessed in the lab, on the job, or both. You may 
assess just knowledge— through a paper-and-pencil 
test, for example. You may assess just attitudes- 
through an attitude scale, for example. Or you may 
assess knowledge, skills, and attitudes simulta- 
neously using a performance test. In the process of 
performing a skill, the learner demonstrates that he 
or she knows the procedures to be followed, that he 
or she possesses such desirable attitudes as safety- 
consciousness, and that he or she can perform the 
skill and produce the product according to specified 
occupational standards. 

Student evaluation vs. piogram evaluation. The 

focus of this module is on student evaluation: gather- 
ing information about student achievement so that 
you know, and they know, how much they have pro- 
gressed since they entered the program and how far 
they have to go to meet the agreed-upon objectives. 
But, as mentioned previously, such information also 
helps you as an instructor to improve your program 
Evaluation results show you where things are not 
working, which allows you to restructure your plans 
so they will work. In addition to making use of stu- 
dent evaluation data for program improvement pur- 
poses, most instructors also use specific program 
evaluation techniques ... but that is not the concern 
of this module.^ 

When to Evaiuate 

Institutional requirements alone should not aster- 
mine your schedule for evaluation activities. Your in- 
structional plans and the learners' need for feedback 
throughout the course will help you decide when to 
evaluate. Generally speaking, there are three ma- 
jor evaluation phases: preassessment, assessment 
of progress, and postassessment. 



1 To gain skill in program evaluation, you may wish to refer to Module 
A-1 1. Evaluate Your Vocational Program, and Module A- 10, Conduct a Stu- 
dent Follow-Up Study. The use cf loliow-up surveys in adult programs is 
covered in Module N-2. Market the Adult Education Program 
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Preassessment involves evaluating learners' skills 
and knowledge in a specific area just before or at 
the beginning of a program (or course or learning 
activity). It enables you to verify student placement, 
identify individual training needs, and determine the 
appropriate level at which to present new learning 
experiences. The results of preassessment also pro- 
vide a reference point for measuring the progress 
learners have made throughout the program. 

In many educational institutions and training pro- 
grams, preassessment is a regular part of the enroll- 
ment and admissions (intake) process. In addition, 
you as an instructor need to ask learners themse'ves 
about their skills, knowledge, and objectives in the 
occupational area. Learners are often the most direct 
source of information about themselves, and they 
can provide insights and information that may have 
slipped through the cracks of the intake process. 

Another preassessment method is to test students 
on the specific content of the program. Theoretically, 
the results can be most useful. By testing students 
directly on the theory, skills, and terminology of the 
program up front, you get a very accurate sense of 
where they are. It can be a reality check for students, 
too. This type of preassessment is particularly useful 
in making pre/postassessmentcompansons to iden- 
tify precisely what and how much learning has 
occurred. 

The disadvantage of this latter type of preassess- 
ment IS that it can start students off with a negative 
experience. Presumably students are in the program 
because it is an area in which they lack skill or 
knowledge. That s a given. If you start by focusing 
on what they don't know, rather than on some suc- 
cess experiences, it is not particularly motivational. 
Thus, if you choose to use such tests, you need to 
administer them in a positive context— focusing on 
the usefulness of the results (1) in identifying where 
students are already strong and (2) in providing the 
information needed to plan relevant instructional ex- 
periences for the specific members of the class. 

Assessment of student progress should be an 
ongoing process and should occur at regular times 
throughout the program. It is an essential part of ef- 
fective learning. Not only do learners gain valuable 
knowledge about their achievements, but experience 
with assessment activities can help them build self- 
evaluation skills that they can use in years to come. 
As the instructor, you need the information gained 
from ongoing assessment in order to provide effec- 
tive learning experiences. 



Information from both formal and informal 
assessments can also help learners identify specific 
weaknesses they might not otherwise discover until 
the end of .he program. At the end of the program, 
It might be too late to acquire important skills that 
they need to meet their objectives. Obtaining feed- 
back dunng the program gives learners a chance to 
develop, practice, and build on these skills. 

You should informally assess student work fre- 
quently and provide students with immediate feed- 
back about their progress. More-formal assessments 
(e.g., tests) should be made on a regular basis. As 
a general rule, the most valid and reliable results are 
provided by tests administered any time from several 
days to two we3ks after SKills and knowledge have 
first been learned. Your teaching situation— the 
length and type of the program, the instructional 
plans you have made, and the nature of the learners* 
activities— will help you determine the best times to 
conduct progress assessments. 

Postassessment occurs at the end of a program 
or segment of a program. In a conventional time- 
based program, for example, exams may be given 
at the end of each unit and at the end of the term. 
Theoretically, the results of final exams sum up what 
students have Ittarned since the beginning of the 
term (i.e., since preassessment). Postassessment 
results often play a large part in determining a stu- 
dent's tinal grade for the term. They also should be 
used as a basis for improving instruction the next 
time the course or program is offered. 

In competency-based education (CBE) programs in 
which learners continue to work on each occupational 
competency until they have reached proficiency, post- 
assessments generally occur more frequently. When 
a student feels he or she ha , mastered a compe- 
tency, he or she performs that skill for the instructor, 
who rates the student's skill level using specific 
occupational criteria. If the student performs success- 
fully, he or she moves on to the next competency. 
In such programs, the student may leave the pro- 
gram, not with a final grade (e.g., >A-F), but with a 
competency profile showing the ratings he or she 
received on each individual competency mastered 
(see sample 1). 

Another form of postassessment may be a part of 
your instructional program. !n some occupational 
areas, such as cosmetology ar,o nuroing, students 
must pass a licensing exam before e.itenng the oc- 
cupation. As with other assessment efforts, you 
should make use of the results of licensing exams. 
The results of such exams give a very clear message 
to you and your students about the adequacy of your 
instruction and the adequacy of their efforts. 



8 10 



CO 




Q SOURCE: Arkansas Deoartment ot Education. Division of Vocational and Technical Education 
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Evaluation Strategies 



There are many different evaluation strategies 
and. as an instructor, you will use a variety of 
strategies throughout the course. In simphfied terms, 
these strategies fall into two basic categories, infor- 
mal and formal. 

informal Evaluation 

During the program, you will find that you are most 
often involved in informal evaluation activities. These 
can include (1) observing students as they learn and 
perform various tasks, (2) talking to individual 
learners about their plans and achievements, (3) 
helping learners find solutions to problems that they 
have, and (4) setting and mainlining an open-door 
policy that encourages learners to approach you for 
feedback. All these activities are crucial to the learn- 
ing process. 

Why are these activities essential to effective 
learning? Learners are not always able to accurate- 
ly self-evaluate, and they need constant, positive 
feedback throughout the course. Lack of experience 
with the subject matter, lack of self-confidence, or 
the inability to always be objective about oneself all 
contribute to a learner's need for feedback from an 
instructor. In each step of the learning process, 
learners need to know that they have achieved or 
can achieve their objectives. 

One of your major roles is to provide learners with 
positive feedback and helpful information about their 
work. Adult learners need immediate information that 
confirms that their work is "correct" and headed in 
the right direction. Praise for the sake of praise is 
not the purpose of positive feedback. It must be 
given for specific achievements, otherwise, it is 
generally meaningless and does little to help develop 
learning. 

No matter how much learners have achieved, you 
should consistently point out and make positive com- 
ments on each learner's achievements. Whether 
they are learning quickly or slowly, all students need 
the encouragement and motivation they gain when 
you identify and respond positively to their 
achievements. Do not neglect "small" successes- 
each of these accomplishments builds toward the 
final objective and should be recognized. 

Likewise, learners need your expert help in iden- 
tifying their mistakes and finding ways in which to 
correct them. But remember, this is useful only if you 
provide the learner with information or help in solv- 
ing the problem. Simply pointing out mistakes serves 



no good purpose and ultimately can discourage 
learners altogether. When learners are having prob- 
lems, you need to ask them questions that will help 
them identify (1) the specific cause of the problem 
and (2) possible solutions. When you approach dif- 
ficulties in this way, you not only help learners with 
the immediate .situation, but you also help them learn 
how to solve problems independently. 

Written comments on projects, papers, and tests 
are another valuable source of feedback. The same 
principles that govern effective oral feedback apply 
to written comments. Make sure that ycu respond 
to specific aspects of the learners* work, as well as 
to the overall product. 

Formal Evaluation 

Formal evaluation involves the use of evaluation 
instruments and grading systems. Using these in- 
struments and systems is generally not extremely 
difficult, but you will need to understand some basic 
principles, steps, and guidelines in order to use them 
fairly and effectively. For this reason, the remainder 
of this information sheet and the one m Learning Ex- 
perience 11 focus on developing effective evaluation 
instruments, designing good tests, and using the 
results effectively. 

Two major approaches to testing are used in most 
adult occupational programs, written tests and per- 
formance tests. Paper-and-pencil or written tests 
consist of objective items, subjective items, or a com- 
bination of the two. Objective items are those in 
which the correct answer for a problem will always 
be the same, no matter who scores the test. Multiple- 
choice, matching, and true-false items are familiar 
examples. Subjective items (e.g., essay items and 
case studies) require the person scoring the test to 
interpret or make a judgment. 
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Whereas written tests are used to evaluate 
knowledge, performance tests require the student 
to demonstrate skill in a particular task or set of 
tasks. The instructor observes the process or prod- 
uct (or both) of the learner's performance and rates 
the student's proficiency accordmg to specified 
criteria. 

When and how you administer tests will depend 
on a number of factors. Some of these factors, such 
as the principles and steps involved in developing 
a good test, will be fundamentally the same in each 
situation. Other factors will vary depending on where 
you are teaching and who your learners are. Let's 
look at the variable factors that will affect the evalua- 
tion strategies you use. 

Institutional factors. Learners may need to gain 
specific knowledge, attitudes, and skills (competen- 
cies) in your program in order to obtain academic 
credit, a license, a certificate, or a degree. To 
demonstrate that they have met these requirements, 
learners may have to pass standardized tests 
developed locally or by a testing service. If so, you 
will need to identify the specific requirements so that 
you can develop learning experiences that will help 
learners meet those requirements and prepare for 
these tests. At the beginning of the program, you 
should also notify students of the expectations and 
requirements of the institution. 

In other situations, the educational institution may 
not provide st=\nda'dized tests but may require you 
to develop teaoher-itiade tests based on explicitly 
stated instructional objectives. These tests are in- 
tended to provide proot to administrators, accredita- 
tion agencies, prospective employers, and students 
that learners have acquired specific skills and 
knowledge in your program. If you are teaching in 
an industnal trar ing program, the firm that has hired 
you may also require this form of proof. 

You will need to be aware of additional institutional 
policies regarding evaluation. For example, you may 
have to submit grades or other types of progress 
reports throughout the program. You should con- 
tact program administrators in order to answer the 
following questions: 

• What kind of evaluation reports must you sub- 
mit, if any? 

• What policies govern the release of student 
evaluation data to others? 

• Are standardized forms provided? 

• What kind of grading system, if any, is used? 
Letter grades? Numerical ratings? Percentages? 

• What levels of achievement do these grades 
represent? 

• What additional evaluation procedures does the 
institution require? 



• What concerns regarding evaluation seem of 
special importance to the institution (e.g., 
dropout rates, grade inflation)? 

Psychological factors. There are a number of 
psychological factors that affect how well a person 
will perform during evaluation. Anxiety, intimidation, 
frustration, fear of failure, and physical discomfort 
can all negatively affect a learner's performance. 
When this happens, the evaluation results don't truly 
reflect a person's capabilities or knowledge. 

Many adult learners who are returning for the first 
time to educational programs have negative 
memories and feelings associated with the idea of 
quizzes, tests, and examinations. For a variety of 
reasons, they may not have performed well on tests 
in the past. Right or wrong, they may still feel that 
when they take lests, they are victims of an arbitrary 
system, which— when the results are in— seals their 
fate. 

As a consequence, the way in which you present 
evaluation activities is an important part of the 
evaluation process. In your role as an instructor, you 
can help generate an atmosphere that says that 
learners are not helpless victims. By providing 
positive and informative feedback throughout the 
course, you can do much to help prepare learners 
for tests, both psychologically and in the subject 
area. 

You should emphasize that tests and other evalua- 
tions are not designed to be a vehicle for pigeon- 
holing students or their learning experiences as 
^'successes" or "failures." Rather, tests provide a 
means for (1 ) determining where students are in rela- 
tion to meeting their objectives and (2) identifying 
areas that need to be strengthened in order for 
them to meet those objectives. Tests are learning 
devices. You should also let learners know that, in 
addition to measuring their abilities, test results 
reflect the quality of the learning experiences that 
you have designed. 

You can do much to help learners become more 
comfortable with evaluation if you openly 
acknowledge their possible anxiety. For example, 
discussing such thoughts as the following may be 
helpful: 

• That they may be experiencing some anxiety 

• That some levels of anxiety and apprehension 
are common and normal 

• That they are not alone in these feelings 

• That anxiety about doing poorly does not 
necessarily foreshadow poor results 

Many instructors have found that group discussions 
about testing and evaluation in general can help 
relieve undue anxiety and dispel self-doubts that 
learners mav have. 
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In addition, when learners have their first test com- 
ing up, you can offer tips on how best to study for 
and complete an exam, including the following. 

• How to space out studying, rather than cram- 
ming 

• How to determine which topics to focus on 

• How to approach the test (e.g., importance of 
reading and following directions, writing legibly, 
reviewing responses for accuracy) 

• How to determine how much time to spend on 
each test item (e.g., knowing to skip an item if 
they do not know the answer immediately and 
then to return to it after they have finished the 
rest of the test) 

• How to use the test results as a learning tool 

Learners' skill and knowledge levels. How well- 
developed are your learners' basic communication 
skills? This is an important consideration. A technical 
knowledge test that requires extensive use of these 
skills will not yield valid results if learners' basic skills 
are not well developed. Students may know the in- 
formation being tested very well, but if they are 
unable to understand and respond to the test items 
as presented, their test results will be poor. 

Students with poor listening and speaking skills 
may have difficulty demonstrating the true extent of 
their technical knowledge on an oral test. Students 
with poor reading and writing skills may have difficul- 
ty demonstrating the true extent of their technical 
knowledge on a written test. You will need to make 
sure, tnerefore, that the test items /ou use corres- 
pond to learners' abilities to understand and 
respond. 

This is not to say that the development and evalua- 
tion of students' communication skills may not be 
a crucial part of your instructional program. It sim- 
ply means that when you want to test technical 
knowledge, you should ensure that you do so— that 
you do not allow other factors, such as weak com- 
munication skills, to cloud your results. 

Special needs. Physical impairments may also 
cause poor test results that don't truly reflect a 
learner's knowledge or abilities. For example, a 
physically handicapped person might need better 
wheelchair access to machinery or v;ork areas in 
order to perform some tasks p 'operly. A visually im- 
paired student might have difficulty reading test 
items in standard-size print. In such cases, you 
would need to modify the evaluation process in order 
to evaluate students fairly.2 



2 To gam more skill tn evaluating the performance of learners who have 
special needs, you rray wish to refer to module L-9. Assess the Progress 
of Exceptjona! Students. 



Individual needs and objectives. Ideally, the 
primary function of evaluation should be to measure 
how well individuals are meeting their own learning 
objectives and gaining skills and knowledge that will 
help them meet their own overall goals. This ideal 
situation does not often exist, however. Other factors 
(e.g., institutionally established learning objectives, 
professional requirements, industrial standards) af- 
fect the functions evaluation must serve. Even so, 
adult learners need feedback that directly relates to 
their individual needs and goals. You should take 
this into account when developing both formal and 
informal evaluation activities. 

Learning domains. For practical purposes, learn- 
ing is often categorized into three primary areas, or 
domains: knowledge (cognitive), skills (psychomotor), 
and attitudes (affective). Understanding the nature 
of these domains and their relationship to different 
types of learning activities can help you determine 
exactly what type of learning you will evaluate and 
the most effective way to do it. Of course, learner 
performance in these domains will overlap, but it is 
useful to consider each of the three domains when 
developing evaluation plans. 

Imagine, for instance, a program on the repair and 
maintenance of automobile brake systems. Before 
students actually perform routine maintenance tasks 
on vehicles, the 'nstructor would probably want to 
test their knowledge of brake systems. A written or 
ora! test could be used to test students' knowledge 
and determine whether they are ready to move on 
to the hands-on learning experience. 
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At certain points in the program, the instructor 
would need to test learners' skills in vanous tasks. 
A performance test that required learners to actually 
change brake shoes, for example, would be an ap- 
propriate method of evaluation for that type of task. 

Learners' attitudes are also an important factor 
in the maintenance of automobile brake systems. For 
example, students need to value and act on a 
number of safety precautions related to working on 
automobiles. They also need to value a properly 
maintained brake system. Thus, the instructor would 
need to evaluate students' knowledge of these safety 
factors, as well as their performance of safety 
measures in actual practice. 

Student performance objectives. All the factors 
discussed so far can play an important role in deter- 
mining the kinds of assessment techniques you use. 
But it i3 student performance objectives that actually 
form the basis for evaluation. These objectives 
define the specific knowledge, skills, and attitudes 
that learners need to acquire in your program, and 
they include the criteria by which student perfor- 
mance should be measured. 

For example, an objective might be as follows: 

G'ven ten shafts with differing measurements, 
measure the diameter of each with a micrometer 
within ± .001" of the instructor's measureirent. 

The objective includes a condition statement (given 
ten shafts with differing measurements), an action 
statement (measure the diameter of each with a 
micrometer), and a criterion statement (withm 
± .001" of the instructor's measurement). 



The cntenon statement describes the standard to 
be met, and in occupational training programs, oc- 
cupational standards generally prevail. Students 
need to learn to perform according to the standards 
they will have lO meet in the job for \which they are 
training. 

Where your objectives come from will be deter- 
mined by your particular instructional situation. The 
school, college, or firm may provide you with a very 
specific list of objectives that must be met in ^our 
program. Or you may need to specify program ob- 
jectives yourself, based on occupational require- 
ments and students' training needs. Or students' 
individual educational and traini.ig needs may play 
a very large role in determining program objectives, 
as may be the case in a continuing education pro- 
gram, for example. 

When the institution specifies objectives, the in- 
stitution should also specify the criteria to use in 
measuring whether the objectives have been met. 
When the objectives are derived from occupational 
requirements, occupational standards should form 
the criteria to be met. When the instructional situa- 
tion is such that students help specify some or all 
of their own objectives, then they must also help 
specify the criteria to be met. 

For example, an individual may enroll in a com- 
puter literacy course for professional development 
reasons, even though computer use is not a require- 
ment of his/her job. In that case, he or she needs 
to help determine the criteria that will define "suc- 
cess," based on his/her o^n personal ob;c:tives. 



Grades and Grading 



Depending on your situation, you may be required 
to submit only records of final grades to program ad- 
ministrators, or you may have to submit more com- 
plete records (e.g., records of progress reports and 
grades received throughout the term, copies of tests 
and performance checklists for each student). You 
Will need to consult with administrators or super- 
visors where you are teaching to determine what pro- 
cedures and what type of grading system (if any) are 
required. 

At the beginning of the program, you should in- 
form learners about the grading procedures and 
standards that will be used. They need to know what 
to expect and how it will affect their overall records. 



Grading Systems 

A number of grading systems are in fairly common 
use. The most common is a letter grade (A S, C, 
D, F) or numerically equivalent grade point (4, 3, 
2, 7, 0) system. Both of these systems are based 
on a five-point scale, in which each letter or number 
is equal to a verbal quality rating. For example: 

A (or 4) = Excellent work, clearly superior perfor- 
mance 

S (or 3) = Very good work, exceeds minimum 
standards of performance 

C(or2) - Work meets standards, acceptable 
performance 

D(or 7) = Below standard work; barely accept- 
able performance 

F(orO) = Work not acceptable 
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Some institutions also provide an / (incomplete) 
grade category. If learners receive an / they must 
complete their unfinished .vork in a set amount of 
time before a final grade is given. 

Another common grading system uses percent- 
ages. The following is one example of percentage 
ranges and equivalent quality ratings: 

90%-100% = Excellent 
80%- 89% = Very good 
70%- 79% = Acceptable 
60%- 69% = Barely acceptable 
Below 60% = Not acceptable 

Percentages allow you to calculate grades more 
precisely because they include a wide range of in- 
crements; you can report not just an >A, but a low A 
(e.g., 90%) or a high A (e.o , 100%). Many instruc- 
tors prefer to assign percentage grades throughout 
a course so that when all grades are averaged, the 
final grade will more precisely reflect the student's 
actual level of performance. Percentage grades can 
then be converted to letter grades or numerical 
grade points, if required by the institution. 

Note that the relationship between letter grades 
and percentages will vary among institutions. For in- 
stance, in some schools, 92-1 CO percent is equal 
to an A, and 84-91 percent is equal to a S, in others, 
an A is equal to 90-100 percent, and a S equals 
80-89 percent. No matter what values you assign 
to the grades you award dunng your program, the 
grades that you submit to administrators should cor- 
respond to the standards and values they have 
established. 

Some instructors and educational institutions use 
an S/U grading system. In order to receive an S 
(satisfactory) grade, a learner's performance must 
meet or exceed specified minimum standards. A U 
(unsatisfactory) grade reflects work that does not 
meet the minimum standards. Pass/fail grades are 
similar in principle to S/U grades. 

There is some controversy over the value of S/U 
and pass/fail systems. Some educators believe that 
these systems are unfair and unrealistic because 
they cannot truly reflect, except at the most fun- 
damental level, the quality of a learner's work. 
Others argue that these systems are superior to 
other systems because they deliver the essential in- 
formation; that is, the learner either possesses or 
does not possess certain skills and knowledge. 



Other institutions report progress and achieve- 
ment, not using grades, but using written evalua- 
tions, skill reports, performance checklists, or 
competency ratings (as in sample 1, pp. 9-11). 
Brevity and clarity are essential to developing fair 
and accurate reports. 

Some instructors, especially in QBE programs, 
use grade contracts. They have learners contract 
for specific grades in each learning experience 
and/or for the complete program. This method is par- 
ticularly appropnate for use with adult learners. 
Adults will generally appreciate, respect, and com- 
mit themselves to a giade contract. This approach 
recognizes their autonomy and ability f choose the 
appropnate level and amount of work for inemselves. 
Sample 2 provides additional detail about contract 
grading. Sample 3 is an example of a variaMe con- 
tract for determining grades. 

If you are teaching in a situation where grades are 
not an issue, you may simply plan regular one-to- 
one conferences and group discussions to provide 
learners with insights and constructive feedback 
about their work. 

Grade Conversion 

If you must submit final grades to the institution, 
you will need to convert all grades earned during the 
program into a single grade. This grade should 
reflect each learner's total achievement. In order to 
convert grades, you must first establish the relative 
value, or importance, of each test or graaed learn- 
ing activity. 
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'f you figure that the total maximum performance 
level for your program is 100 percent each test or 
graded learning activ-ty then constitutes a percent- 
age of the total possible grade For example, grades 
in one program n^Jght he v;eighted as follows: 

60% - Six in-ciast projects Q 10% each 

20% - Two written papers @ 10% each 

1 0% * Two take-home assignments ^ 5% each 

5% - Mid-term examination 

5% - Final examination 

lOO^A = Total possible grade 

Of course, in every learning situation, the relative 
value of assignments will vary. The most Tiporint 
concern is that the items be weighted in a way that 
reflects the true value oi each individual activity in 
relation to the whole learning experience. If you 
are teaching a situation in which major learning 
objectives and projects can be individualized, 
learners may have some options about how grades 
are weighted. Regardless of how the relative value 
of .earning experiences is established, however, 
learners need to be made aware of these values at 
the beginning of the program. 

Using the established values, you can then con- 
vert each learner's scores to a final grade. Sample 
4 uses the preceding weighted grade values to 
demonstrate how an instructor converted one 
learner's grades to a final grade. The column on the 
left shows the learner's grddes. The column on the 
right shows the instructor's conversion calculations 
(with totals rounded off to the nearest tenth). The 



subtotals on the nght are added lo determine the 
final grade for the course. If necessary, the final 
percentage grade could be converted to a letter 
grade based on established letter grade/percentage 
equivalents. 

Grade Records 

!t is vital that you koep accurate and up-to-date 
records of all grades for each learner. These records 
serve several purposes. You will refer to them for 
information about learner progress and 
achievement— information you need to assist in- 
dividual learners and plan relevant instruction for the 
class. Your records provide the documentation you 
need to calculate final grades for reporting purposes 
and to justify decisions made concerning the award- 
ing of credits, licenses, or degrees. These records 
can also be useful when students or former students 
ask you to write letters of recommendation for thei • 

The actual system that you choose for recording 
and storing grades is, to some extent, a matter of 
personal preference. The "best" method will be one 
that v^orks for you. Many educational institutions pro- 
vide instructors with record books designed ex- 
pressly for keeping grades. Some instructors prefer 
to use other materials, such as index-card files or 
!oose-!eaf binders, in designing a record-keeping 
system for their grades. 

Regardless of the record-keeping system that you 
choose, it is vital that you keep these records in a 
safe place to ensure learners' privacy. 
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SAMPLE 2 




SOURCE Excerpted from Module D-5. De'-^mme Student G ades, p 12 
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SAMPLE 4 



Ihtiructor^s Grade Caicu 



" .:88%" ' 

-,l9(W<)\..;'; 

.85,%;'= ■ 



'■''■V 



. / Twoiwrittih i^imi''^: l6%«iich 



; '89,%;-' 



9-5 

MiX:;;^^?*^ 8.8 

96^x' iibi- i9;b;' 

85;x.,;ll »; 8;5; 
■.95iicUl6;«.\9;5: 



.84.x-;iO 8:4: 

";92^;.rid-». ^g^; 



. , ' . 90 =x .05 » 4.5 

. • ' ■ .-„'?5.x,',65..-. ;4.8 . ■ 

\ 86 . x .05 ^ii-. 4;3% 

- ' ' ' - 89 X .05,' = 4;5%. 

LEAFiNER!S .FiNAL GRADE FOR COURSE = 90.0% 
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Using Evaluation Results 



No evaluation process is really useful unless the 
results of assessment are reviewed, analyzed, and 
incorporated into plans for future learning. Conse- 
quently, it is important that you do more than sim- 
ply mark and record learners' grades, return the 
graded tests and assignments to individuals, and 
move on to new learning experiences as originally 
planned. 

Both you and the learners need to review assess- 
ment results and identify the following: 

• Individual and group achievements 

• New learning experiences for which learners 
are now prepared 

• Ways that newly acquired skills and knowled;^ . 
can be used in upcoming learning experiences 

• Skills or knowledge areas that need to be 
strengthened 

• Resources (including the learners' strengths), 
techniques, and activities that learners can use 
to reach competence in their weaker areas 

• Changes that need to be made in the instruc- 
tional plans In order to help learners meet their 
objectives 

It is important to remember that positive evalua- 
tion results don't just point to learners' successes. 
They also indicate your success in enabling others 
to meet tJieir objectives. You should take time 'o 
recognize when your own work is well done— when 
you have provided effective instruction and an en- 
vironment that is conducive to learning. 

Remember, too, that the causes of poor evalua- 
tion results (i.e., low grades) need to be identified 
before they can be addressed > These results can in- 
dicate a variety of things. For example: 



Learners' nonreadiness for svaluation— 
People learn at different rates— those who do 
poorly one week may be prepared by the next. 
All tests do not have to be given at tiie same 
linre. The objective of a program should be 
learning— not test laking at a fixed time. 
Mi^assessment of learners' training 
needs- When leamtrs' training needs and 
Skills are not identified (or not identified ac- 
curately) during preassebsment, the result may 
be poor po-iormance on evaluations. For exam- 
ple, some students may need remedial instruc- 
tion in certain areas. It is important ♦o discover 
this as early as possible, so that individuals can 
gain the skills they need to participate in the 
learning experiences. 

Weaknesses in the instructional plan- 
Perhaps learning activities began at too high or 
too low a level, given what learners already 
knew and could do. ln?truction may need to be 
modified or paced dirterently. Bear in mind that 
when activities are paced too slowly, some 
students will become bored and inattentive. 
Thus, they may do as poorly on tests as learners 
who find that learning ©y^eriences are moving 
too quickly. 

Unrelated problems— Sometimes problems 
not related to instruction affec< evaluation. For 
example, iliness of a relative, unemployment, 
marital difficulties, and a mynad of other fac- 
tors (including an instructor's negative attitude 
toward the course or lack of genuine concern 
for the students) can affect learners' perfor- 
mance. 



ERLC 



21 



P6 



^ Optional 



Activity 



V 3 / 



For more information on student evaluation and testing in general, you may 
wish to read one cr more of the following supplementary references. Morris 
and Fitz-Gibbon, Wow to Measure Achievement; Popham, Modern Educa- 
tional Measurement; Gronlund, Measurement and Evaluation in Teaching; 
and/or Kirkpatrick, Wow to Improve Performance Through Appraisal and 
Coaching. 



m Optional 
S Activity _ 



To gain additional skill in conducting stuc' it evaluation and grading, you 
may wish to refer to one or more of the following PBTE modules: 

• Module D-1: Establish Student Performance Criteria 

• Module D-5: Determine Student Grades 

• Module K-3: Organize Your Class and Lab to Install CBE 



M Activity J 



The following items check your comprehension of the material in the infor- 
mation sheet, Student Evaluation, pp. 6-21 . Each of the three items requires 
a short essay-type response. Please explain fully, but briefly, and make sure 
you respond to all parts of eacn item. 



SELF-CHECK 

1. During a discussion about teaching adults, one instructor flatly states, "Adult learners should never be 
evaluated, and if vou must assign grades because the institution requires it, you should just give one 
easy test at the end of the »^ourse." What would be your response to this instructor? 
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2. The following seven factors can affect the evaluation process. Read each factor, and describe how it 
may affect the evaluation strategies you use. 



• Institutional factors 



Psychological factors 



^ • Learners' skills and knowledge 'evels 

I 
I 



• Special needs 




• Individual needs and objectives 



• Learning domains 



• Student performance objectives 
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3. What is the best type of grading system to use? 
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Compare your written responses to the self-check Items with the model 
answers given below. Your responses need not exactly duplicate the model 
responses; however, you should have covered the same major points. 



MODEL ANSWERS 

1 . You might well disagree with the instructor, citing 
the following as reasons in favor of evaluating the 
performance of adults. 

Typically, adults are utility-oriented learners who 
bring specific needs and objectives to a learning 
situation. Their needs may be immediate, and 
given other life responsibilities, the amount of 
time and energy that they can commit to a learn- 
ing experience may be limited. 

As a result, adults want and need proof that their 
needs are indeed being met— whether they are 
in forma! or independent learning situations. Ef- 
fective evaluation activities not only provide 
learners with this information, but also help adults 
learn to assess their own performance more ac- 
curately. Tangible proof of progress provides a 
vital boost to the morale, thus, it reinforces and 
encourages learning. 

As the instructor, you also will need the informa- 
tion provided through evaluation. By incor- 
porating assessment activities into c II phases of 
the program, you can take into account learners' 
strengths, weaknesses, and objectives and 
design instructional plans that will enable learners 
to meet their needs. 

Furthermore, one easy test at the end of the 
course would not provide you, the learners, the 
institution, or licensing and certifying agencies 
with valid information about learners' real abilities 
and achievements. Effective evaluation occurs 
during all phases of the program, at the begin- 
ning of the program (preassessment), throughout 
the program (assessment of progress), and at the 
end of the program (postassessment). 

2. Institutional factors. Learners may be required 
to gain specific skills, knowledge, and attitudes 
in order to receive a degree, academic credit, a 
license, or a certificate. They may have to take 
standardized tests based on criteria specified by 
an educational institution, business/industry, or 



certifying or licensing agency. Learners need to 
be prepared for this, and your assessment ac- 
tivities must take these criteria into account. 

The way in wh'ch grades are to be reported and 
other types of evaluation requirements may also 
be part of institutional policies. In some cases, 
report forms are provided for instructors, and a 
schedule for submitting evaluation reports is 
fixed. There might be an established grading 
system that reflects specific levels of achieve- 
ment. Your grading system will need to be com- 
patible with the institution's evaluation policies 
and procedures. 

Psychological factors. Some adult learners 
have negative memories of tasting. For any num- 
ber of reasons, they may not ',ave performed well 
and, hence, may be anxious about being tested. 
Even those who have performed well on tests in 
the past may still be anxious in a testing situa- 
tion. If the fear is great enough, it can negatively 
affect a learner's performance and yield test 
results that do not reflect the learner's true 
abilities. 

If anxiety is a factor, you need to emphasize the 
ways in which assessment activities can help 
learners. Through identifying their strengths and 
weaknesses, learners can determine where they 
are in relation to their objectives and strengthen 
skills as needed so as to meet those objectives. 

You should also openly acknowledge test-anxiety 
and its commonness. Your reassurance can go 
a long way in helping learners (1) feel more com- 
fortable wtth testing and (2) discover positive 
techniques for dealing with evaluation. 

In addition, you can help learners cope better with 
testing by providing them with tips and strategies 
for test-taking. 
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Learners' skill and knowledge levels. Learners* 
skill and knowledge levels affect how you develop 
a test and write test items. For example, a test 
becomes less valid if it requires students to use 
communication skills they don't have. When this 
happens, learners cannot demonstrate their 
knowledge because they cannot communicate it 
to you. Thus, it is essential that you identify 
learners* skill and knowledge levels and develop 
tests appropriate for those levels. 

Special needs. Learners may have special 
needs that require modification.^ in the testing 
process. For example, a visually impaired student 
might have difficulty reading a written test. A 
physically impaired student might have problems 
in fitting written responses in the spaces provided 
or in operating equipment that is not modified for 
his or her use. Any such needs must be provid- 
ed for in the evaluation and testing situation. 

Individual needs and objectives. Adult learners 
bring their own individual needs and objectives 
to a program. These are often as important as 
your overall program objectives and should be 
identified and included, in individualized form, in 
assessment activities. 

Learning domains. Learning is often categorized 
into three basic areas: knowledge (cognitive), skills 
(psychomotor), and attitudes (affective). In 
evaluating student performance, you need to en- 
sure that you evaluate learning in each area, as 
appropriate. 



For instance, knowledge may be assessed via 
written tests. Skills may be assessed via perfor- 
mance tests. And attitudes (e.g., regard for safety, 
punctuality) may be assessed via written tests and 
classroom observations. Learning in all three 
areas may be measured— directly and indirectly— 
via a performance test. 

In each situation, you need to identify (1) the type 
of learning covered by the objective(s) and (2) the 
most logical and efficient way to assess learners' 
achievement in that domain or those domains. 

Student performance objectives. Just as stu- 
dent performance objectives define the knowl- 
edge, skills, and attitudes to be learned, they also 
establish a basis for evaluation. As you identify 
objectives to be covered in the program, you 
should also identify the specific criteria by which 
learners' achievement should be assessed. The 
source of the objective (the institution, the oc- 
cupation, or the learner) should alao be the 
source of the criteria. 

3. There is no one best grading system that you 
should use in all situations. Your particular 
situation— the requirements of the institution in 
which you are teaching, course objectives, and 
learners' objectives— will determine the type of 
grading system you should use. 



Level of Performance: Your written responses to the self-check items should have covered the same ma- 
jor points as the model answers. If you missed some points or have questions about any additional points 
you made, review the information sheet. Student Evaluation, pp. 6-21, or check with your resource person 
if necessary. 
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# Learning Experience II 



OVERVIEW 




Effective evaluation includes the use of effective tests, and in most cases, 
Activity will need to develop your own. For this you will need some knowledge 

\ M o\ the principles of test development. For a general introduction to the dif- 

1 ferent types of tests, their uses, and how to develop them, read the follow- 

^^^t^^ ing information sheet. 



HOW TO CONSTRUCT A TEST 



Knowledge about who, what, and where you are 
teaching is not enough information on which to 
develop a test. In order to develop an effective 
evaluation system, you will also need to know (1) the 



characteristics and uses of good written and perfor- 
mance tests and (2) the guidelines for constructing 
different types of test items. Let's look first at writ- 
ten tests. 



Written Tests 



Written tests may be objective or subjective, they 
must be valid, reliable, and usable. Sample 5 pro- 
vides some basic information on these three prin- 
ciples of test development. 

Objective vs. Subjective 

As noted in the sample, whether you use objec- 
tive or subjective test items in a given situation can 
affect the reliability of your test— its ability to con- 
sistently measure what it is supposed to measure. 
Each type of item has distinguishing features, ad- 
vantages, and disadvantages. 

Objective items. These are items in which the 
correct answer will always be the same, no matter 
who is giving or scoring the test. The following are 
some major advantages in using objective test 
items: 

• Objective test items can usually be answered 
fairly quickly. Therefore, you can test for a fairly 
wide range of information in one test. 

• These items can be scored relatively quickly, 
easily, and accurately using an answer key. 

• Judgment calls and opinions are neither 
necessary, possible, nor desirable in scoring 
these test items— hence their reliability is high. 

The disadvantages associated with objective test 
items include the following: 

• Objective test items can be time-consuming to 
develop and write. 

• They are limited in the types and levels of learn- 
ing that can be evaluated. They test mainly the 
learners' ability to identify or recall information, 
not their ability to analyze, synthesize, and 
organize their knowledge. 



• Even professionally developed test items may 
be ambiguous, and learners are often given no 
oppoilunity to justify or qualify their answers. 

• Even the best-written objective items can 
sometimes allow learners to guess the correct 
answer. 

Subjective items. These items require the scorer 
to jUdge and interpret individual responses to each 
Item. In order to ensure that a subjective test is as 
valid and reliable as possible, you will need to 
establish an objective answer key that includes all 
the major points that learners should have covered 
in their responses. 

The following are some distinct advantages to the 
appropriate use of subjective test items. 

• They can test learner's ability to analyze, syn- 
thesize, and organize their knowledge; and they 
can cover a broad or narrow range of informa- 
tion. 

• Subjective test items provide learners with a 
means for creative problem solving. There are 
many cases in which there may be no single 
correct or workable solution to a problem. A 
suDjective item may be most appropriate in 
these situations. 

There are two primary disadvantages in the use of 
subjective test items: 

• More time is needed for readir.g and scoring 
these items than for scoring objective items. 

• The validity and reliability of a subjective-item 
test may be lower than that of an objective-item 
test. Since the value of an answer depends in 
part on how well it is defended and supported, 
it may be hard to judge its value in absolute 
terms. 
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SAMPLE 5 




fus^ljlty^ of'items as m\\: 

rt^woUtd'ffiakeia test 
rsltuationj 



SOURCE: Excerpted from Module D-2, fKtBBtt Student Performance: Knowledge, pp 8-10 
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Experienced teachers have noted that the objec- 
tive test is usually broad, but thin— including wide 
ranges of subject matter and many items, but never 
at any particular depth. The reverse is true for the 
subjective test; it is usually narrow, but deep. Hence, 
many teachers consistently provide tests that include 
both types of items, not infrequently in something 
approximating a 50-50 mix. 

Planning The Test 

Test design— including both the format and the 
way in which the instructions and items are 
written— can also affeci the testes validity, reliability, 
and usability Sample 6 outlines important pro- 
cedures for developing and administering a good 
written test. As noted in the sample, careful planning 
is a key element in developing a good test. The 
following are a few simple steps for planning a test: 

• Determine when you will need to administer the 
test. 

• Identify the specific student performance objec- 
tives to be covered by the test. 

• Identify specific criteria, or standards, by which 
learners* achievements will be measured. 

• Determine the type(s) and number(s) of test 
items that are most appropriate for evaluating 
each kind of performance (knowledge, skills, 
attitudes). 

• Develop the test items. 

• Determine the relative value of the objectives 
and the criteria, and establish the point value 
of test Items accordingly. 

• Pian how you will present the test to the 
learners; include such information as why you 
are giving a test, the skills and knowledge areas 
that will be covered, whether the results will be 
part of a final grade, and if so, what percentage 
of the total grade they will constitute. 



Developing Objective Test Items 

Four major types of written test items are generally 
considered to be objective, multiple-choice, match- 
ing, true-false, and completion items. 

Multiple-choice items. In multiple-choice items, 
the learner selects from a list of possible responses 
the one correct (or most nearly correct) answer to 
a problem or question. These items are especially 
useful for evaluating learners* knowledge, including 
their ability to recall, identify, and synthesize a wide 
range of information in a single test. 

Multiple-choice items provide a simple response 
format for the learner and quick, accurate scoring 
for the instructor. Initially, you may find multiple- 
choice items somewhat time-consuming and difficult 
to write. But many teachers feel that the advantages 
in administering and scoring them more than com- 
pensate for this. You will also find that, with ex- 
perience, good multiple-choice items become easier 
to write. 

The problem or question stated at the beginning 
of the item is called the stem. The possible answers 
to the item listed are called responses. The correct 
response is simply referred to as the answer, and 
the incorrect responses are referred to os distracters. 

The stem can be stated as a problem to be solved, 
a question to be answered, or an incomplete state- 
ment to be completed. By varying the stem, you can 
adapt the multiple-choice format for your particular 
use. Sample 7 shows examples of multiple-choice 
items with various stem formats. 

Regardless of the stem format you choose, stems 
must be clearly and unambiguously stated. Students 
cannot be expected to provide a correct answer if 
they cannot clearly understand the question or the 
problem. 

When devising responses, remember that not 
only must all disiracters be completely incorrect, but 
they must also be plausible. If distracters are par- 
tially correct, the test is not a valid or reliable 
measure of the learners* knowledge. Likewise, 
distracters that are not plausible are a dead 
giveaway to the correct answer. This also results in 
a test that is not valid. 

It is equally important that the answer be com- 
pletely correct. If students know ihe information, or 
how to solve the problem, there should be no doubt 
about the correctness of the answer. 
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SOURCE' Excerpted from Module D-2, Assess Student Performance, Knowledge, pp 10-15 



The way the responses are written should not pro- 
vide clues to the answer. All responses should be 
phrased in such a way that, when linked with the 
stem, they result in complete, comprehensible, 
grammatically correct statements. All responses 
should be of approximately the same length. Inex- 
perienced test writers often write longer correct 
answers, with more qualifiers, in order to ensure that 
they are completely true. The noticeable difference 
in length is a clue to the correct answer. 

A response pattern is another clue to the correct 
answers. When the correct answers always appear 
in the same position in the response list (e.g., the 




answer is always in position B) or if they run in a 
predictable pattern (e.g., B-E-C, B-E-C, B-E-C), the 
result is a re?:ponse pattern. Students generally 
become alerted to this and find that they can rely 
on the pattern, rather than on their knowledge, to 
supply the correct answer. 

The optimum number of responses for a single 
item IS usually four or five. With this number, the 
chances of guessing the correct answer are lower 
than if there were fewer responses. More than this 
number of responses can be confusing or take too 
much time to review. This can cause the learner to 
lose sight of the point of the question. 



SAMPLE 7 




Matching items. Matching Items require learners 
to identify significant relationships— between words 
or concepts, between principles and their applica- 
tions, or between symbols and objects, for example. 
These items can be fairly easily constructed. 
Because they require learners to do little writing, 
matching items can be administered and scored 
relatively quickly. 

Matching items are generally composed of two 
lists. One list (usually the one on the left) is made 
up of premises, while the other contains responses. 
A relationship is stated between the two groups, and 
the learner is asked to match each piemise with the 
correct response. Sample 8 shows a portion of a 
matching item, including directions. 

When developing matching items, you should pr;,- 
vide approximately 10 to 15 elements in each list. 
(Dealing with more items tends to be confusing for 
the learner, and providing fewer items encourages 
the learner to guess.) In addition, you should always 
provide more responses than premises in order to 



help reduci the possibility of guessing the correct 
answers by process of elimination. One list, usually 
the premises, should be sequentially numbered, and 
the other, alphabetically lettered. This helps to avoid 
confusion in both answering and scoring. 

Because students need to read the list of 
responses more than once in order to identify the 
correct response for each premise, the responses 
themselves should be brief. The premise statements 
need not be short, but they should be clear and 
unambiguously stated. 

When producing (e.g., typing) the test, be sure that 
the whole matching item appears on a single page. 
It is time-consuming and unnecessarily confusing for 
learners If they have to look back and forth between 
two pages to view all possible responses to each 
premise. 

As with other types of test items, you will need to 
safeguard against response patterns that can help 
learners predict the correct responses. 
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SAMPLE 8 
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True-false items. On true-false items, learners 
must identify whether each given statement is true 
or false. You can use these items to assess a wide 
range of factual knowledge, as well as learners' abil- 
ity to interpret and summarize this knowledge. 

True-false items do not require learners to do ex- 
tensive v/riting and therefore can bo administered 
and scored fairly quickly. A problem with true-false 
items, however, is that they are often less reliable 
because students have a 50-50 chance ot guessing 
the right answer. 

Statements for true-false items are presented in 
list form. Students are usually asked to mark their 
responses to the left of each statement. Responses 
can be marked in various ways (e.g., by writing true 
or false or by circling Tor F), as shown in sample 9. 

True-false statements should be specific, brief, 
and clear. Learners must know precisely what the 
statement is saying before they can determine 
whether it is true or false. Each item must also be 
either completely true or completely false. If items 
are partly true and partly false, learners have to 
guess whether they are to respond to the true or the 
false portion of the statement, and the test is no 
longer valid. 

The validity of true-false items will also be reduced 
if the wording provides clues or if learners have to 
guess at your definitions. For example, absolute 



words, such as always, never, completely, all, more, 
and totally, tend to be associated with false 
statements and thus provide clues to the answer. 
Ambiguous words, such as usually, frequently, some, 
often, and several, require learners to guess at your 
personal definition of these words. How often is fre- 
quently? How much is some? Be specific when citing 
measurements of time, volume, o'* other quantities. 

Some instructors give in to the temptation to lift 
direct quotes from textbooks or lecture notes when 
writing true-false statements. Although this may 
simplify the instruct *s task, it does not produce a 
valid means of assessing learners' understanding 
or knowledge. Rather, it tests learners* ability to 
memonze what they have read or heard. You can 
develop good true-false items by focusing on the 
essential information covered in your course. 

True-false items can be developed in ways that 
are more thought-provoking and less a guessing 
game. In contrast to what was said earlier, items can 
be developed so that some are partially true and par- 
tially false— if the directions specify that **if any part 
of a statement is false, mark it false.** Or instead of 
simply being asked to identify whether a statement 
IS false or true, learners can be asked to correct any 
false statements. The third example in sample 9 
shows this latter type of modified true-false item. 
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SAMPLE 9 




^<fal^;:indlcw^^ item ;ls^ 
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Completion items^ A statement in which a key 
term or phrase is omitted and replaced by a blank 
space is a completion item. The learner must com 
plete the statement by supplying the correct word 
or phrase. 

These items can be used to assess learners* 
recall of a wide range of facts, information, and prin- 
ciples, rather than their ability to simply identify a cor- 
rect answer in a group of provided alternatives. 
Because the answer is not supplied, the guessing 
factor is lower than with matching, multiple-choice, 
and conventional true-false items. Like these other 
types of items, however, completion items can be 
scored fairly quickly and easily. 

You can begin to develop completion items by 
writing a sel of brief statements that :over the most 
important information included ir- t^e course or in- 
structional unit. The next step, as you review each 
statement, is to identify a key term, symbol, or 
numerical value; delete it from the sentence; and 
replace it with a blank. If an article or other word that 
would provide a clue to the answer (e.g., a, an, this, 
the, these) precedes the blank, you should omit that 
too. Note that only significant terms should be omit- 
ted from completion items. 



Items should provide enough specific information 
so that students can know how to respond. This is 
perhaps best illustrated by the following example of 
how not to write a comp!et»on item. 

The moons of the planet Jupiter were first 
discovered in 

Learners could provide a variety of correct 
answers to this statem^ >t: 7670 (the year in which 
the planets were first discovered); a specific area of 
the rnght sky (or, for that matter, the sky itself); orb/7 
around Jupiter; Italy (the country in which Galileo, 
the discoverer, lived); and so on. 

Depending on the knowledge that you are testing, 
this item could be better stated as follows: 

The moons of the planet Jupiter were discovered 
in the year . 

The moons of Jupiter were first discovered when 

the planet's position was at degrees of 

arc. 

The astronomer who discovered the moons of 

Jupiter was living in the country of at the 

time of the discovery. 
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To ensure that the item is both reliable and valid, 
there should be only one possible correct answer 
for each term that is omitted. When scoring comple- 
tion items, it is important to remember that the word- 
ing of correct responses need not be identical to 
those in your answer key, as long as the meaning 
is the same. 

Although more than one key term may be omit- 
ted from a statement, the item should require only 
brief, factual answers e., short lists of words, sym- 
bols, or numbers). If you want to determine whether 
learners can supply and discuss a body of informa- 
tion, essay items are more appropriate. 

Regardless of the length of the answer, blank 
spaces for all items should be the same length so 
as not to provide clues to the correct term. To sim- 
plify both the answering and scoring of completion 
items, some instructors insert only small blanks m 
the body of the statement itself and provide larger 
blanks for the responses in a column to the left of 
the item. 

Developing Subjective Test Items 

The most common type of subjective test item is 
the essay item. Essay items require the highest level 
of communication skill and call for the most complex 
thinking patterns of all the test items discussed so 
far. They can be a valuable tool for assessing 
learners* ability to organize, analyze, synthesize, and 
present related information. They can also be used 
to test learners' ability to solve problems. 

Depending on the specificity of the essay ques- 
tion or statement, you can cover a fairly wide or nar- 
row range of subject matter in one test item. You can 
require responses that vary m length from a single, 
short paragraph composed of one or two sentences, 
to a few short pages of prose. 

A potential drawback to the use of essay items is 
that learners who have a low level of verbal com- 
munication skills aie at a disadvantage. Hence, 
essay items may not provide a valid or fair test of 
their knowledge or analytical abi-'ties. If such writ- 
ten communication skills are not ci itical to vSuccess 
in the occupation, essay items are proi^ably not an 
appropriate assessment technique 

Both the administration and scoring or essay items 
can be quite time-con&uming. Students need more 
time to develop responses than they do for shorter 
test items. Sconng, too, takes longer because it re- 
quires careful reading, interpretation, and judgment 
on the part of the scorer. 



Types of essay items. One of the most impor- 
tant factors m developing an essay stem is determin- 
ing precisely the knowledge that you will evaluate 
and the type of essay iten, that is most appropriate 
for assessing that knowledge. Common types of 
essay items include su.nmary, critical analysis or 
critique, comparison and contrast, and case study. 
Each type has somewhat different applications. 

You might request a summary if you want to as- 
sess learners' ability to recall, identify, and report the 
essential elements of a body of information. For ex- 
ample, in a studio photography course, a summary 
essay item might be stated. Summarize the advan- 
tages of using a moderate telephoto lens in portrait 
photography. 

You night ask learners to provide a critical 
nalysis or critique when it is important to deter- 
mine whether they can identify, analyze, and weigh 
the strengths, weaknesses, limitations, and unique 
characteristics of a particular method or tool. An ex- 
ample of this type of essay item might be stated, 
Wnte a critical analysis of the uses of the 35mm lens 
/;; landscape photography. 

You might use a comparison and contrast essay 
when you want learners to identify and discuss the 
similarities and differences m the uses of various 
principles, methods, tools, or machinery. An exam- 
ple of this kind of essay item might be. Compare and 
contrast 35mm and 2%" format cameras. 

If you need to test learners* ability to apply knowl- 
edge, you can develop a case study or recommen- 
dation essay item. This is an item in which learners 
are asked to analyze a situation or problem ar»d. 
based on their analysis, recommend an effective 
solution. These items should be presented in the 
form of case studies that describe realistic cir- 
cumstances (o.g., a supervisor's role in dealing with 
tardy employees, a malfunctioning automobile that 
exhibits certain symptoms). 

Case study items can also take the form of video 
presentations. Learners view videotapes of various 
situationr and then propose solutions. No matter 
how you choose to present the case study, the cir- 
cumstances piesented shOt-*d be realistic and pro- 
vide enough detail that learners have relevant and 
sufficient information to work with. 

Constructing items. After you h: identified the 
appropriate type of essay item, the next step is to 
develop it The essay item should be stated fully and 
opecificaliy. clearly outlining (1 ) the vype of informa- 
tion or solution that learners must provide and (2) 
how learners should present that information. 
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You should identify all the factors that are being 
c sessed as part of the item. Learners need to know 
specifically what elements will be scored and the 
weight or point value of each one. These elements 
can include content, organization, grammar, punc- 
tuation, spelling, clarity of the writing, and others. 
Remember also that it is not legitimate to score any 
factors that are not part of the learning experiences. 

Many instructors recommend that communication 
skills be scored separately from the content. In this 
way, both the instructor and the learners can distin- 
guish between mastery of the subject matter and 
quality of communication skills. 

Scoring. You will need to develop a scoring key 
that identifies all of the information and other fac- 
tors that are being tested. The next step is to deter- 
mine the point values of the item as a whole and of 
each factor. With all these data cle'^rly outlined, you 
will have a comprehensive scoring Key that will help 
minimize the subjective element in scoring essay 
items. 

In order to ensure that the item is as objective and 
valid as possible, you should compare the item with 
the answer key and ask yourself these questions. 



• Is the item stated in such a way that it clearly 
requests the responses that are listed in your 
key? 

• Do the instructions clearly state the way in 
which learners are to respond? 

• Given the lengl.i of the test, is there sufficient 
time for learners to fully respond to the item? 

Bear in mind, however, that because each \c^i ler 
will organize and synthesize information differently, 
flexibility is a key element in scoring essay items. 
You will find that, with some practice, you can 
develop an answer key that !s comprehensive and 
specific, yet flexible enough to allow for variations 
in individual responses. Many instructors find that 
it is easiest to be objective and evenhanded in their 
scoring if they read all learners' responses to a single 
essay item in one sitting. 

If there are learners in your program who are in- 
experienced with essay items or whose basic com- 
munication skills are not well developed, you may 
want to plan practice sessions during which they can 
develop these skills. Practice sessions and tests 
should begin with short essay items that require less 
writing and less complex organization and build 
gradually to longer items. 



Performance Tests 



There are many situations in which pencil-and- 
paper tests are not the most appropriate evaluation 
technique. Performance tests can provide a practical 
means of assessing how well learners can (1) carrv 
out a process or sequence of tasks and/or (2) create 
a product. 

Performance tests can be a valuable tool in a 
number of phases of instrjction. They can be used 
as pretests to determine learners' current skill levels 
before beginning the program or a new learning ex- 
perience. (One caution: Pretests of a performance 
nature should never be used if potentially dangerous 
equipment, materials, or operations are involved.) 

Performance tests can also be used as progress 
assessments, as final assessments, and as a part 
of the actual learning experience. Learners can use 
these tests as a self-check in practicing their skills 
and determining their own levels of achievement. 
This is a common feature of competency-based edu- 
cation (CBE) programs. 

In developing performance tests, it is important to 
remember that they must conform to the same high 
standards as written tests: they must be valid, 
reliable, and usable. 



Performance tests (see sample 10) generally in- 
clude three major components: 

• A clear statement (directions) concerning the 
testing situation (What task is the learner ex- 
pected to perform? What product is he/she 
expected to produce? How will the performance 
be evaluated?) 

• A checklist of criteria, or performance stan- 
dards, to be met 

• A rating scale 

Directions/Testing Situation 

The first step is to identify the objective or objec- 
tives to be covered by the test. If properly stated, 
each objective should specify the action to be per- 
formed, the conditions under which performance 
must occur, and the criteria to be met. for instance, 
a health care program for hospital aides might in- 
clude the following objective: 

Objective: Given a hospital bed occupied by a pa- 
tient who is dependent on mechanical life-support 
systems (condition), change the bed linens (ac- 
tion) without injury to the patient (criterion), while 
maintaining the life-support link at all times 
(criterion). 
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Based on the objective(s), you next need to 
describe the situation in which the learne/c perfor- 
mance will be evaluated. The situation should In- 
cluo'v^ specific details about the task(s) to be per- 
formed and the learner's role. For example: 

Situation: As a hospital aide, you are changing 
bed linens. Your next patient is confined to bed 
and dependent on mechanical life-support equip- 
ment. You need to change the linens on that pa- 
tient's bed. Based upon the criteria listed below, 
your instructor's evaluation will determine whether 
you are competent in this task. 

In some cases, the situation you devise will be an 
actual one. In this particular case, where the 
learner's errors could be life-ihreatening to a real pa- 
tient, you would need to devise a simulated situa- 
tion. For example, another learner could role-play the 
patient. 

Criteria/Checklist 

What criteria, or standards, will you use to evaluate 
the learner's performance? Will you evaluate the 
learner's performance by observing only the process 
followed . . . only the product produced . . . both? 
What other elements (e.g., time, attitudes) need to 
be evaluated? 

You may evaluate process only m situations 
where no tangible final product is produced (e.g.. 
Greet visitors to the office). You may evaluate prod- 
uct only, which saves time, when all key process 
elements can be inferred from observing the final 
product. In other words, the only way the learner 
could have arrived at an acceptable final product 
was by following the correct process. Frequently, you 
will want to evaluate both process and product. 

A finished product provides valid evidence of pro- 
ficiency because it is readily measured, and the mea- 
surement can be done with relative objectivity. If you 
are evaluating a product, you will need to list all the 
factors or elements that should be present and ob- 
servable in the product. For example, product criteria 
for the linen-changing objective might include the 
following: 

• The patient was uninjured by the linen-changing 
process 

• The life-support systems were still in place 

• The linens on the newly made bed had tight 
hospital corners 

Process, too. must be evaluated m situations such 
as the following: 

• Critical inner parts of a final product are hidden 
from view (as in a surgical dressing, where inner 
aseptic measures and medications are of even 
greater importance than the outer visible areas) 



• Sanitation or safety procedures are of oxtreme 
importance and must be meticulously observed 

• Evaluation involves "live customer" work (as in 
cosmetology, where an error in the hair dyeing 
or permanent process could have long-lasting 
unhappy consequences) 

For example, steps involved in the linen-changing 
process might include the following: 

• The learner adjusted the bed to the proper 
height 

• The learner removed the old linens carefully 
from the bed 

• The learner communicated reassuringly with 
the patient 

• The learner ensured that life-support systems 
were always in place 

Often, attitudes are crucial to occupational skills. 
And they're not as impossible to measure as is 
usually supposed. Measurement just requires a lit- 
tle attention and thought on your part. You need to 
determine what behaviors would indicate the 
presence of the desired attitudes. 

We all say things like, "He*s a very courteous per- 
son" or "She's a real leader.'* How do we come to 
those conclusions? We get clues from a person's ac- 
tions: he says please and thank you; she volunteers 
for extra work, and her peers ask her for help. 

Some skills involve important time factors that 
must be measure j. The batter must be beaten for 
five minutes. The typisi must be able to type 45 
words per minute to achieve entry-level competence. 
Time standards may come from specific industry 
documents, such as flat-rate manuals. Or they may 
derive from general trade agreements and union 
contracts. 

From your own experience and expertise in the 
field, you will be able to identify many of the criteria 
for successful performance. Other sources for iden- 
tifying criteria can include employers, labor unions, 
adviLo,7 committees, and manufacturers of the 
equipment or matenals that are being used. 

Tnese criteria, logically sequenced and clearly 
slated in observable terms, form the basis for the 
performance checklist. The checklist should have at 
least the following important qualities. 

• The checklist should be short enough to make 
it practical for you to use. Perhaps 5-10 items 
are sufficient for a simple skill, 13-20 items (at 
the most) for complex skills. 

« The criteria included must be critical to suc- 
cessful performance of the skill. Minor or trivial 
criteria just make your evaluation job more dif- 
ficult and time-consuming. 
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• Each criterion should have some qualitative 
base. It is not enough to record that the student 
did something; he/she may have done it poorly 
or very well, and this needs *o be shown. 

• The items must be simple and unambiguous— 
quickly read and understood by you and the 
students. 

• Items should be stated in parallel terms (e.g., 
all in the past tense). 

If you find that your checklist is impractically iong, 
review the criteria and determine whether each item 
is truly essential. If the checklist cannot be short- 
ened, you should consider dividing the original per- 
formance objective into two separate objectives that 
can be assessed individually. 

Rating Scales 

There are a number of types of rating scales that 
you can use. The type you select and the format you 
use will depend on the naiure of the performance 
to be evaluated. 

Yes/no rating scale. One kind of rating scale is 
the simple yes/no scale (see sample 10). The learner 
either did or did not perform the task at an accep- 
table level, and you mark yes or no accordingly. No 
assessment is made of the degree to which the per- 
formance was acceptable. This type of system is 
most appropnate when increments of quality are not 
an issue. 

Let's say, for example, that a performance test re- 
quires the learner to change the blade on a Ve-inch 
table saw. One of the criteria states. The nut on the 
bolt must be completely tightened, and there is no 
acceptable, observable range of tolerance. In this 
case, a yes/no scale would be appropriate. 



Multi-level rating scales. In many instances it is 
important to assess the relative quality of the per- 
formance or product, or the degree to which a char- 
acteristic is present. Furthermore, there may be 
broad tolerances for successful performance in 
some areas. In any of these cases, it ^s most ap- 
propriate to use a multi-level rating scale that in- 
cludes a range of performance levels. 

There are two basic types of multi-level rating 
scales. Probably the most commonly used is a 
numerical scale, in which numbers (e.g., 1 to 5) 
designate various levels of quality for each factor that 
is assessed. It is important that the level of quality 
represented by each number be clearly defined and 
explained to learners. The following is a list of stan- 
dard equivalents: 

5 = Excellent 
4 = Very good 
3 = Good 
2 = Fair 
1 = Poor 

Some instructors prefer to use verbal response 
categories that contain evaluative words, rather 
than numbers. The Teacher Performance Assess- 
ment Form (TPAF) at the end of this module is an 
example of this type of rating scale. Whether you use 
numerical or verbal rating scales for evaluation, five 
levels of performance seem to provide the optimum 
range of responses. 

Format. Rating scales can be arranged in a vari- 
ety of formats. Among the most popular approaches 
is to place columns of blanks or squares to the right 
of the performance items (criteria). At the top of the 
scale, the rating categories are listed from left to right 
(e.g., 1,2,3, 4,5, or their verbal equivalents). Directly 
below each category are the columns of blanks or 
squares. When each item is evaluated, a check mark 
is placed in the appropriate blank or square. The 
TPAF in this module and the checklist in sample 10 
use this type of format. 

A graphic scale is composed of a horizontal line 
that has response categories marked along the line. 
Each item on this type of checklist has its own in- 
dividual scale, and ratings for each item are placed 
in the appropriate place. Graphic scales are par- 
ticularly useful when brief, task- or product-specific 
descriptions of performance levels for each item are 
necessary. Sample 1 1 shows a graphic rating scale 
for one item. 

Levels of ratings. In rating a learner's perfor- 
mance, there are at least two levels involved. You 
need to rato performance in terms of each in- 
dividual criterion, or item, on the checklist. You also 
need to rate the individual's overall performance. 
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SAMPLE 11 




If you look at sample 1 0, pp. 37, for example, you 
will note that a simple yes/no rating scale has been 
provided for rating each criterion. However, at the 
top of the performance test, there is a multi-level 
scale (1-5, with clear descriptors provided for each 
level) for rating the learner's overall level of 
performance. 

Administering the Test 

Before you administer a performance test, you 
should provide students with copies of the checklist 
to use as a reference while practicing their skills. In 
CBE programs, performance checklists are often a 
part of student learning guides, modules, or other 
instructional packages. If it is at all possible in your 
situation, you should ask learners to determine when 
they are ready to have their performance formally 
evaluated and set a convenient date with you. 

When prepanng for the test, it is your responsi- 
bility to be sure that all the necessary materials, 
tools, and equipment arc available. The actual area 
in which the test will take place should be clean and 
free of distractions. 



It is best to administer process-oriented tests in- 
dividually because you will need an unobstiucted 
view of each step in order to evaluate the learner's 
perforn.ance fairly. In some instances, it mciy be 
possible to administer the test to a small group of 
learners at one time, provided that (1) you have a 
clear view of each person's activities and (2) 
everyone in the group is prepared for the test at that 
time. 

If product evaluation is the focus of the test, 
students may simply submit their products to yo j for 
evaluation. 

After the performance test is completed, you must 
determine whether each student's performance as 
a whole meets at least the minimum acceptable stan- 
dards. If not, the learner will need to practice in order 
to bring performance up to acceptable standards. 
Regardless of the total performance rating, you 
should schedule follow-up conferences with learners 
to discuss how and why you have arrived at your 
assessment. 
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/ Optional 
' Activity ^ 

V 2 J 



For more information about how to construct tests and test items, ^ ou may 
wish to read one both of the following supplementary references: 

• Denova, Test Construction for Training Evaluation, is a readable manual 
that provides much useful information on evaluation, including clear 
directions on how to develop an evaluation plan, tests, and individual 
test items. 

• Mager and Beach, Developing Vocational Instruction, discusses, step 
by step, the processes involved in developing instruction. Chapters 5, 
6, 7, and 8, pp. 28-51, explain how to establish course objectives and 
prerequisites, and identify types of performance and appropriate mea- 
suring instruments. 



M Optional 
* Activity _ 



To gain additional skill in developing tests, you may wish to refer to one or 
more of the following PBTE modules: 

• Module D-2: Assess Student Performance: Knowledge 

• Module D-3: Assess Student Performance: Attitudes 

• Module D-4: Assess Student Performance: Skills 

• Module K-4: Provide Instructional Materials for CBE 

• Module K-5: Manage the Daily Routines of Your CBE Program 
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■ Activity J 



Read the following test developed by a vocational instructor. As you read, 
keep in mind the principles and steps involved in developing an effective 
test. After completing your reading, critique in writing all essential factors 
of the test. 



UNIT TEST 




Unit Test: Business Letters 



Directions: The following is a test that covers the complete learning unit on business lette^'o. It includes 
both a written and performance test. The performance test will be administered at the end of the time allowed 
for the written test. 



True-Fa!se 

1. In a full-blocked btyle of business letter, the date l.ne, complimentary closing, and wnte:'s identification 
all begin at the center. All other lines begin at the left margin. 



2. In the semiblocked style of business letttr, the date line, complimentary closing, and writer's identifica- 
tion all begin at the center. The first line of each paragraph is inaented five spaces. 



3. The simplified style of business letter includes the following features All lir es begin at the left margin, 
a subject line replaces the salutation, punctuation is open, the complimentary closing is deleted, and 
the writer's name and title are typed on one line in all capital letters. 



4. In the blocked style of business letters, all lines begin at the left margin. 
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Matching 

Directions: The list on the left provides definitions of various parts of a business letter. The list on the right 
contains the names of the various parts. Match each name with its correct definition. 



5. 


the writer's name or position or both 


a. 


letterhead 


6. 


the writer's and/or typist's initials 


b. 


date line 


7. 


indicates the way in which the letter was sent 


c. 


inside address 


8. 


a closing phrase, such as "Yours truly" 


d. 


salutation 


9. 


the name and address of the person who is receiving 


e. 


message 




the letter 


f. 


complimentary closing 




10. 


indicates that the letter is written to a specific person 




company signature 




or aepaiimeni 


g. 


11. 


an opening phrase, such as "Dear Ms. Morris" 


h 


Willci o lUcI 1 U llOallUII 


12. 


the names of people other than the addressee who will 


i. 


reference initials 




receive the letter 


j- 


mailing notation 


13. 


a businesses printed name and address 


k. 


carbon copy notation 


14. 


the month, day, and year that the letter is typed 


1. 


return address 


15. 


the typed name and address of the business, if letterhead 


m. 


subject line 




stationery is not used 






n. 


attention line 


16. 


tiie text of the letter 






17. 


used in place of salutation; typed on a separate line in 








all capital letters 






18. 


indicates that the writer speaks for the company 







Completion 

Directions: rill in the blanks. 

19. The heading of a business letter includes and . 

20. The opening of a business letter includes and . 

21. The body of the business letter includes 

22. The closing of the business letter includes , and 

Multiple-choice 

Directions: Each of the items below is foiiowed by four possible responses. Select the one correct response 
for each item. 

23. If a building name is included in the inside address, it should: 

a) be typed in the same line as the street address 

b) be typed in the line directly below the street address 

c) be typed in the line directly above the street address 

d) be typed in the sarre line as the addressee's name 
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24. House and building numbers in street addresses should be: 

a) typed in numerical figures 

b) typed out in word form 

c) typed in the same line as the street name 

d) typed preceding the street name 

25. Which number used as a street name should be spelied out? 

a) 24 

b) 100 

c) 17 

d) 4 

26. The U.S. Postal Service abbreviation for the state of Maine is: 

a) MN 

b) ME 

c) MA 

d) Ml 

27. The U.S. Postal Service abbreviation for the state of Maryland is: 

a) MA 

b) ME 

c) MN 

d) MD 

© Essay 

28. Your toxt describes eight ways to lengthen a short letter. List and describe all of them. 

Performance Tesl 

Learner's Name: Date: 

Directions: Given access to tht noec'ed office suDpiies, edit and type a handwritten draft of a business 
letf . , using pnnted letterhead stationery, 12-p;t^^. t^pe, and the blocked style. Your performance will be 
assessed using the following criteria. The instructor will check the YES or NO box to indicate whether you 
performed each step correctly or whether the prodjct rnet each criterion as indicated. 

YES NO 

The learner: 

1. selected appropriate stationery D D 

2. identified all errors (e.g., spelling, punctuation, capitalization) in the handwritten draft . . . 

3. corrected all errors accurately □ □ 

4. made corrections without changing the meaning of the letter n □ 

5. used appropriate margins and spacing given the type of stationery, the size of the type, 

and the length of the letter D D 

6. completed the task in V2 hour or les?> □ □ 
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The 

7. 



completed letter: 

included all standard parts in the heading, opening, body, and closing 
was laid out in accordance with standard block stv'e 



□ 
□ 
□ 
□ 



□ 
□ 
□ 
□ 



8 



9 



was error-free 



10 



was neat and clean (e.g., free of smudges) 



Number of criteria attained 

Number of criteria not attained 

Total score 

Instructor's signature 

Level of Performance: All items must receive a YES response. If any item receives a NO, the learner and 
instructor should meet to determine what additional activities the learner needs to complete in order to reach 
competency in the weak area(s). 



• 
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Compare your written critique of the test with the model critique given below. 
Your response need not exactly duplicate the model response, however, you 
should have covered the same major points. 



MODEL CRITIQUE 



Although this test has its strong points, many sec- 
tions need to be thought through and revised before 
It is administered to learners. 

Some of the sections, as well as individual items, 
are clearly written and easy to follow. But others are 
so poorly and ambiguously stated that learners have 
to guess at the intent of questions or statements in 
order to respond. This does not create a valid, fair, 
or reliable test. 

Test directions are inconsistent. Again, some are 
clear and direct while others are incomplete, unclear, 
or simply absent. The general directions at the begin- 
ning of the test, as far as they go, are fairly well writ- 
ten. However, they are incomplete. These directions 
should include information about (1) the total point 
value of each section of the test and (2) the amounts 
of time allowed for the written and performance por- 
tions of the test. Directions for each type of item on 
the test are, in general, poorly written. Specific in- 
stances wili be discussed in relation to the various 
types of test items. 

The test is fairly clearly laid out, readable, and 
easy to follow. There are, however, some distinct 
mechanical problems in the layout (e.g.. in the true- 
false section, no space is provided for learners' 
responses). 

In addition, there are too many different types of 
items in this test. Becau'^e of this, lea;ners have to 
switch mental gears frequently in order to respond 
to different types of item3. This ca.i make it difficult 
to concentrate on the comc.'^t of the test. Perhaps 
some types of items could be rewritten in another 
format. F'^- instance, the compbtion items might be 
writte'' . short essay items and grouped with the 
other essay item (e.g.. Item 22 could be stated: 
Name the three basic elements that form the closing 
of a business letter). 

Although the;8 are too many different types of 
items on the test, the number of items seems to be 
too few, given the intendeo scope of the test. And 
many of those items that are on the test cover a nar- 
row range of intormation. Test directions clearly state 



that the te^. will cover a complete unit on business 
letters. Thus, the learner might assume that a wide 
range of knowledge and skills is to be tested. How- 
ever, the test does not do this, and the result is a 
spotty and sometimes trivial assessment of the 
learners' knowledge. 

The multiple-choice section, for example, could be 
expanoed to include more items with a greater vari- 
ety of content. As it is, it deals only with a few aspects 
of one fe'>ture of the business letter— the address. 
Two of tnese six items— those that deal with U.S. 
Postal Service abbreviations— simply test learners' 
rote memorization of information, which is probably 
less critical than other knowledge that minht be 
tested. 

In addition to the general strengths and 
weaknesses of the test, the individual sections of the 
test have both weak and strong points. These are 
addressed individually in the following discussions. 

True-False 

The two main problems with this section are tnat 
there are (1) no directions for how the item is to be 
answered and (2) no spaces in which to write 
responses. Complete directions, including the point 
value of individual items, should always be provided 
for each type of item on the test. 

The individual items seem to be stated clearly ^nd 
specifically enough that they are either true ur false. 
However, the instructor may have provided clues to 
the correct answers by writing longer, more detailed 
statements for trua items than for those that are 
false. Maintaining a fairly uniform lengih of true-false 
items IS an important factor in developing items that 
do not help learners guess the correct answer. 

This portion of the test is short— not a problem in 
itself. However, this section would have been a good 
place to include a greater number and variety of 
items to better cover the scope of the unit. 
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Matching 

Generally speaking, the matching section is one 
of the better developed parts of the test. It com- 
prehensively covers an area of knowledge. Both the 
premises and responses are clearly written, and 
there is no predictable response pattern that pro- 
vides clues to the correct answers. 

On the other hand, the possibility of guessing is 
increased by having exactly the same number of re- 
sponses as there are premises. There should always 
be more responses than premises. And Item 15, as 
well as the directions for the performance test, pro- 
vides a clue to the correct response to Item 13 
{letterhead). 

Another problem with this section is that the direc- 
tions do not explain how to marK responses and no 
spaces are provided for responses. The directions 
are otherwise clearly stated, but this is an important 
omission. 

Completion 

This is one of the most poorly developed sections 
of the test. The few directions given do not include 
the point value of the items and do not explain the 
statement. Fill in the blanks. With what? 

All the items are written so ambiguously that 
learners could supply a number of different correct 
answers for each item. For example. Item 21 could 
be correctly answered with such responses as 
words, information, tt xt, message, and many others. 
Items like this turn test-taking into a guessing game 
and produce tests that are neither reliable nor valid. 

Another problem with the completion items is the 
different-sized blanks. The lengths could provide 
clues to correct answers (in this particular case, 
however, the statements are so ambiguous that 
learners need all the help they can get!). 

Multiple-choice 

The instructor did remember to write some direc- 
tions for this section, and what is there is understand- 
able. But once again, the directions are incomplete. 
They do not indicate how learners are to mark their 
responses, nor is the point value of the items staled. 

Overall, the items are clear and well written. No 
clues to the correct responses are provided in the 
stems, and there is a good number of responses to 
choose from. 



One exception, which lowers the quality of this 
section and reduces the validity of the test, is Item 24. 
two of the responses provided are correct. Since the 
directions state that learners are to select the one 
correct response, v/hich are they to choose? 

Earlier criticisms of this section—the triviality and 
limited scope of the questions—are also points for 
improvement. 

Essay 

The essay item competes with the completion sec- 
tion for being the worst element of the test, it is stated 
in such a way that it \i. ..:erL ly a test of learners' rote 
memorizaiicn of information provided in the tex\ It 
might indeed be important to determine whether 
learners know how to lengthen a short letter. 
However, the wording of the question implies that 
learners must reproduce memorized, printed infor- 
mation, rather than answer from their own bank of 
knowledge. 

In addition, directions are incomplete: the approx- 
imate length of the required response is not in- 
dicated, nor is the point value of the item cited. Also, 
very little space is provided for the response. Even 
if the instructor wanted a brief answer, there is still 
not enough room, given the range of variation in how 
learners word their essays and how large they write. 

Performance Test 

The performance test is the best portion of the unit 
test, although it would have been more usable had 
it been put all on one page. (This would also have 
allowed nore space for the response to the essay 
item.) It IS clearly laid out, complete, and easy to 
follow. The directions here are the best on the test. 
They are fully and clearly stated, as is the objective. 

The items on the checklist are well written and in- 
clude all important steps and criteria. It should also 
be noted that only important steps and criteria are 
itemized. This contributes to the manageable length 
of the test. 

The yes/no rating scale for the items seems ap- 
propriate for the objective, and the checklist is easy 
to use, both for marking and for interpreting 
responses. 

A space for additional comments is the one ele- 
ment that IS missing from the test. Otherwise, the 
performance test seems complete. 



Level of Performance: Your written critique of the test should have covered the same major points as 
the model critique. If you missed some points or have questions about any additional points you made, 
review the material in the information sheet, How to Construct a Test, pp. 28-40, or check with your resource 
person if necessary. 
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Learning Experience III 



OVERVIEW 







9' 



ERIC 



49 



.^4 



Adults need to be involved in determining both what they learn and how their 
Activity learning will be evaluated. The skills they gain in identifying appropriate 

m evaluation techni-^jes will enable them to become better lifelong learners, 
^ both in and outside of the classroom. For information on how to develop 

s^^^^ evaluation strategies that are particularly appropriate for use with adult 
learners, read the following information sheet. 



EVALUATING ADULTS: WHAT'S APPROPRIATE? 



When adult learners come to the classroom, many 
of them bring a history of independence and respon- 
sibility in coping with life situations. They also have 
a future that demands that they continue to learn and 
be able to make effective judgments in many new 
circumstances. Because of this, it i? vital that you 
devise and implement evaluation strategies that can 
help students lea.n hov/ to realistically identify 
evaluate their own resources, abilities, and knowl- 
edge. 

In many classrooms situations, learners do not 
help determine how and what they will iearn, or how 
and on what basis they will be evaluated. Unfor- 
tunately, this situation does not provide learners with 
essential skills in evaluation. Evaluation strategies 
for adults are most effective when traditional authority 
roles are de-emphasized, and the learner's role as 
an autonomous, responsible adult is emphasized. 
This can happen when learners are actively involved 
in Identifying and establishing their own evaluation 
techniques— techniques that will enable them to 
meet their objectives. 

Admittedly, there will be learning situations in 
which neither you nor the learners will have many 
options in establishing learning objectives or select- 
ing the types of evaluation devices, criteria, or 
grading system to be used. Even so, there is still 
much that you can do to help learners gain more skill 
and autonomy in dealing with what may first appear 
to be arbitrary demands. One of the best strategies 
Is to use collaborative approaches. 

Using Collaborative Approaches 

Collaborative approaches involve learners in iden- 
tifying and developing learning objectiyes, evalua- 
tion techniques, and criteria. These approaches 
acknowledge that adults have not only a right but 
a responsibility to take a more active role in deter- 
mining all phases of their learning experiences, in- 
cluding evaluation. 

Collaboration has several advantages. It en- 
courages a high degree of learner involvement and 
commitment, both from individuals and from the 
group It can enable learners to gair valuable 
understanding of how they learn. 




Collaborative approaches provide a valuable 
source of preassessment information, since learners 
must identify their current skills and knowledge in 
order to determine their learning objectives and 
subsequent evaluation techniques. An additional 
benefit of collaboration is that evaluation is often less 
threatening to learners when they are involved in 
designing the evaluation strategies than if the strat- 
egies are generated solely by an outsiae authority. 

Depending on your situation, the rosults of a col- 
laborative approach can range frorn an evaluation 
system that is totally learner-designed and -scored, 
to one that includes many standardized or instructor- 
designed and -scored tests, papers, and projects. 
Collaboration may not be appropriate In all types of 
learning situations, however. For example, you may 
find that it is too time-consuming and cumbersome 
a process if time in your program is short and 
learners are unfamiliar or uncomfortable with 
developing their own learning and evaluation 
strategies. The extent to which you and learners can 
collaborate will depend on a number of factors, as 
follows. 

Institutional requinments. You may be teaching 
in a situation in which the instructional elements— 
objectives and evaluation tools— are determined by 
a variety of external sources. In an industry-based 
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training progrdm, for instance, the firm supporting 
the program may have established both the objec- 
tives to be met and the evaluation instruments to be 
used. Or licensing requirements for a given occupa- 
tion may govern what objectives must be met, and 
a standardized licensing exam may provide the final 
basis for assessment. Learners seeking to enter a 
new occupation or to reach a higher occupational 
level will need to meet objectives covering the com- 
petencies required to meet those career goals. 

There may be nothing that you can do to change 
these requirements. However, you and the learners 
can still collaborate in identifying additional learn- 
ing objectives and specific evaluation techniques 
that will help them to meet their unique personal 
objectives. 

individual needs. In some situations, both the 
learners and the instructor have numerous options 
in determining the content and methods of evalua- 
tion. In these cases, evaluation can be based largely 
on learners' individual needs, skills, and objectives. 

Imagine, for example, a tvping course in which 
learners are enrolled for different reasons. One is 
a pharmacist who has enrolled to become a more 
accurate touch-typist. Another is employed as a 
typist but wants greater skill in typing statistical 
ta^es. 

After a brief discussion, the instructor discovers 
that accuracy, rather than speed, is the pharmacist's 
main concern. So they develop a learning objective 
based on that need: Touch-type 30 words per minute 
at 100% accuracy. In the case of the professional 
typist, the instructor and learner develop the follow- 
ing objective: Type three different types of statistical 
tables at 45 words per minute, with 90% accuracy. 

Learners' experience. Learners' familiarity with 
different types of learning situations will also af- 
fect the amount of involvement they can have— at 
least initially— in making instructional decisions. 
Learners who are familiar only with conventional, 
instructor-designed courses may be uncomfortable 
with a collaborative approach. At first, it may seem 
like an overwhelming amount of new work. Thus, it 
is important to introduce collaborative activities in 
small, manageable portions. You will need to take 
the time to teach them how to do*"?;op appropriate 
learning objectives and select appropriate evalua- 
tion strategies. 

Learners who are experienced with collaborative 
approaches will probably be more comfortable with 
the process and able to move more quickly in iden- 
tifying learning objectives and relevant evaluation 
strategies. 

Students' experience with the subject area will also 
affect how much they can contribute to the process. 



Learners who have little knowledge of the subject 
area will require much more direction in identifying 
and establishing options m learning and evaluation. 
This will be less of a factor in an advanced course, 
in whicn learners are quite familiar with the prin- 
ciples, techniques, and tools of the occupation. 

Establishing the Basis for Evaluation 

With all these variables in mind, how can you work 
with learners to establish a basis for evaluation? In- 
structors have made use of a variety of techniques, 
tailored to their situations. Three collaborative ap- 
proaches that are especially appropriate for use with 
adults are group decison making, learning con- 
tracts, and grading contracts. 

Group de 3ion making. As the name implies, 
this process ir /olves class members, as a group, in 
identifying and selecting the learning objectives to 
be met and evaluation strategies to be used. Using 
this technique, you can also allow learners to incor- 
porate individualized learning and evaluation 
strategies into the group plan. 

Although learners should have a fair amount of 
freedom in this process, you will need to provide 
some information and structure so that they will have 
a frame of reference for decision making. One 
strategy is to provide forms on which learners can 
identify the following: 

• Learning objectives 

• Evidence of accomplishment (proof that the ob- 
jective has been met) 

o Evaluation strategies (how the evidence will be 
gathered: a written lest, oral test, or perfor- 
mance test; instructor review, peer review, or 
self-evaluation) 

• Criteria for evaluation 

• Timing of evaluation (when the evidence will be 
gathered) 

You will also need to discuss with them any institu- 
tional, occupational, and degree/licensing re- 
quirements that will affect the evaluation system to 
be developed. 

Group decision making may not be appropriate in 
all learning situations. If you are teaching in a course 
in which learners' needs and goals vary greatly, they 
may find it difficult or impossible to reach consen- 
sus about evaluation strategies. Or, if their group 
communication skills aie not well developed, the 
group ''consensus*' may not truly reflect the needs 
and objectives of all group members. Some may 
dominate discussion, while others say nothing. You 
will therefore need to determine whether group doci- 
sion making is really an appropriate means of deter- 
mining evaluation strategies in your situation. 
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The following is a four-step approach that is com- 
monly used in group decision making: 

1 Brainstorm— Learners form committees (of up 
to about seven persons), which meet to gener- 
ate possible (a) learning objectives, (b) evidence 
of accomplishment, (c) evaluation strategies, 
(d) criteria, and (e) timing of evaluation. 
Although this process is designed to enable 
learners to make their own decisions, they may 
need your help. During the committee session 
you could visit each committee to ensure that 
the meeting is running smoothly and produc- 
tively. Learners may have questions that need 
to be answered before they can reach any 
decisions. 

By the end of a set amount of time, learners 
must reach a consensus about all these fac- 
tors. The committee then selects a represen- 
tative to report the group's recommendations 
to the rest of the class. 

2. List— At the close of the committee process, 
the class reconvenes. Each committee repre- 
sentative presents recommendations to the 
class. In your role as a facilitator, you can help 
generate a positive, productive session by 
recognizing the value of everyone's ideas and 
accepting all recommendations as being open 
to discussion. You should list all recommenda- 
tions on a flip chart or chalkboard. 

3. Discuss— After all recommendations have 
be^n submitted, class members discuss each 
recommendation in terms of its ability to meet 
individual and group learning needs. You may 
need to direct the discussion in order to keep 
participants focused on the topic and ensure 
that all members have the opportunity to par- 
ticipate. You may also need to respond to 
recommendations on the basis of your occupa- 
tional experience and to suggest items that 
may have been overlooked. 

4. Decide— When the pros and cons of all recom- 
mendations have been discussed, you and the 
class must reach consensus about instructional 
objectives and specific evaluation strategies for 
the course. 

Learning contracts. The learning contract can 
help learners to clarify their objectives, document 
their learning and evaluation plans, and commit 
themselves to the work they have contracted to do. 
Contracts also provide a fair and clear basis for 
evaluation, since the learner knows exactly what will 
be evaluated and when and how it will happen. 

The amount of learner input into the contract can 
vary greatly, depending on how structured the pro- 
gram is and how independent the learners are. In 
some long-term programs, for example, learners 



who are experienced with the subject matter may be 
required to develop contracts that include some 
learner-designed evaluation instruments. 

In most other situations, however, modified learn- 
ing contracts are used. For example, an instructor 
might develop a variety of learning objectives, learn- 
ing experiences, and evaluation activities. Evalua- 
tion activities might include teacher-graded papers 
and projects, tests, self-evaluations, and peer 
evaluations. Learners would then select from those 
options and contract to participate in specific learn- 
ing and evaluation activities that would enable them 
to meet their objectives. 

There are three preliminary steps that you need 
to take before developing learning contracts: 

1. Assess group and individual training needs. 

2. Present general course objectives and institu- 
tional requirements related to evaluation. Identi- 
fy both negotiable and nonnegotiable objectives 
and evaluation activities. 

3. Discjss with learners ' jw evaluation relates 
to learning objectives. 

Once you are satisfied that learners understand 
these basic principles, you and they can work 
together to develop individual contracts for a given 
time period. The following are steps in developing 
a learning contract: 

1. Identify learning objectives (which ones and 
how many). 

2. Identify learning experiences that will enable 
the learner to meet these objectives. 

3. Determine what will be accepted as evidence 
of accomplishment (e.g., a finished pioduct, 
written test results, learner's performance, a 
paper, a presentation to the class). 

4. Establish criteria for evaluating each item of 
evidence. 

5. Identify appropriate evaluation strategies: How 
will the learner's achievement be evaluated? 
Who will evaluate the learner's work— the in- 
structor, the learner, peers, experts in the field*? 

6. Set a target date for accomplishment of objec- 
tives and a date for evaluation to take place. 

Together you and the learner will have to review 
all the elements of the contract before a final agree- 
ment is made and signed. Your experience in the 
field will enable you to judge whether the contract 
is complete and realistic. You may need to suggest 
items that the learner has overlooked and identify 
possible problem areas in the sequence or schedul- 
ing of activities. When agreement is reached about 
all the elements of the contract, you both s;gn and 
date the contract, and each of you keeps a copy for 
reference. Samples 1L and 13 are examples of learn- 
ing contracts. 
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Throughout the program, you should meet with 
learners individually on a regular basis to review the 
contracts in light of tiie progress they are making. 
You will need to determine jointly if adjustments 
need to be made to the original contract and, if so, 
where and how they need tc be made. At the end 
of each contract, you will need to meet with individual 
learners to determine whether the contract has been 
fulfilled. 

Grading contracts. Grading contracts provide 
learners with options (1) in the relative weight of 
evaluation activities (as in sample 3, p. 19) or (2) in 
the amount of work they will perform. With this 
information— and with knowledge of their needs, 
resources, and abilities— learners can help deter- 
mine how they will be graded. 

In the first situation, a learner who typically does 
very well on quizzes, but whose note-taking skills 
leave much to be desired, could contract to have 
quiz grades constitute a higher percentage of the 
final grade. The learner's grades on individual ac- 
tivities must still be earned, but the relative weight 
of those grades is determined, at least in part, by 
the student. 

In the second situation, the final grade is det r- 
mined by contract. The contract specifies how much 
work must be completed for ar» A, how much for a 
B, and so on. Because of the outside responsibilities 
typical of adult learners, some oA your students may 
have limited time available for study. Those students 
may need to take your class to meet the re- 
quirements established for a degree or by their 
employer. However, they may wish (or have time) 
to do only the minimum work required to obtain a 
passing r^ade; those students could contract to com- 
plete the work required for a C. Other students who 
have the time, the resources, and the need to com- 
plete more work or more in-depth work, could con- 
tract to complete the worK required for a 6 or A. 

Regardless of the grade contracted for, the stan- 
dards to be met must be nonnegotiable. A student 
should not earn the C he or she contracted for simply 
by completing the work; the work must be completed 
so that it meets the precisely measurable standards 
established for academic achievement or occupa- 
tional competence. 

To develop grading contracts, you will need to 
work with learners to specify the learning objectives 
and level of performance they must achieve. Gen- 
erally, learners will need two to three weeks' ex- 
perience in t*^e program before they have a basis 
for developing such contracts. After signing the con- 
tract, both you and the learner should retain a copy 
for reference. 



Selecting Appropriate 
Evaluation Techniques 

Regardless of how or by whom the evaluation 
strategies are established, it is essential that the 
evaluation procedures selected are appropriate to 
each learning objective. 

Consider, for example, an automobile repair 
course in which learners prepare for entry-level 
employment. Occupational standards would provide 
evaluation criteria and suggest the appropriate form 
of evaluation. For the learning objective Gap spark 
plug tc industry specincations, the instructor would 
probably use a performance test. 

If It were important to verify that learners had 
knowledge of the operating principles of the inter- 
nal combustion engine, a cognitive written test 
could be the most appropriate evaluation instrument. 

In many situations, self-as'3e<?Sinent is an effec- 
tive evaluation tool, particuiarly for adult learners. 
Learners can use answer keys to assess their own 
performance on written tssts; they can use perfor- 
mance checklists to rate their competence on per- 
formance tests. Self-assessment can be used in 
preassessment, to help learners identify their skill 
and knowledge levels prior to beginning new learn- 
ing experiences. It can also be used within learning 
experiences, to give learners an opportunity to prac- 
tice and sharpen their skills before taking graded 
tests. 

Through self-assessment, learners can complete 
and score tests in private, without the fear and anx- 
iety often associated with tests given by an instruc- 
tor. Then, when they feel they are ready, they can 
approach the instructor for a graded test. Initially you 
will cften need to help learners determine their 
readiness for the graded test. 

There are some commercially available self- 
assessment tests that can assess a range of skills 
in a variety of occupations. You may wish to check 
with counselors at your institution or state curnculum 
centers in order to determine what is available in 
your field. You can also develop your own self- 
assessment devices, using either performance tests 
or cognitive pencil-and-paper tests. These should be 
accofiipanied by a scoring key or model answers, 
as used in this module, that learners can use to score 
their own work. 

Another excellent means of self-assessment is the 
journal, or log, in which learners record their learn- 
ing plans and activities. Journals are especially 
tioefui in helping students document their learning 
activities and determine what they need to do in 
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order tc meet their objectives. They can be used in 
conjunction with learning contracts or as an indepen- 
dent learning and evaluation strategy. Depending on 
your situation, you may ask learners to submit their 
journals to you for review and discussion, or you may 
choose to have them keep private journals for their 
own personal reference. 

Journal-keeping styles usually depend on in- 
dividual preference. Some learners prefer to keep 
a simple log in list form and to jot down their learn- 
ing activities, the objectives they have achieved, their 
own performance » ^.tings, and their plans for meeting 
new objectives. Others prefer to keep a prose jour- 
nal in which they discuss these things in more depth. 
There are a number of possible variations on these 
basic approaches that learners can adapt to their 
own personal styles. 

If you are teaching in a program in which grades, 
licensing, academic credit, and institutional re- 
quirements are not an issue, y^u can determine 
evaluation devices and criteria largely on the basts 
of individual learners' needs and abilities. But those 
needs may vary quite a bit. Some adult learners may 
still want and need to use a variety of tests that will 
give them specific feedback about their progress. 
Others will prefer an approach which provides them 
with feedback but which is much less formal and 
structured. 

In such a situation, ycu could give the test? that 
the first group desires and a!so provide for informal, 
less structured strategies for the others. Informal 
one-to-one conferences, group discussions, and pri- 
vate journal writing would probably meet the needs 
of the second group. Note, however, that in this type 
of learning situation, learner participation in test tak- 
ing and other evaluation activities should be volun- 
tary. 

Selecting the Appropriate Evaluator 

Who should evaluate the performance of adult 
learners? The learner? The instructor? Employers? 
Workers? Learners' peers? The answer to this ques- 
tion will vary with the situation. In many instances, 
all these people may be involved as evaluators at 
different phases of the learning experience. But 
many educators contend that involving the learner 



in self-assessment activities provides the most ef- 
fective form of adult evaluation. Thus, it is important 
that you build as many self-assessment activities into 
the evaluation plan as possible. 

Self-assessment enables students to learn how to 
form objective judgments and promotes a greater 
sense of self-direction and responsibility. Therefore, 
it helps them gain valuable skills that they can use 
throughout their lives, in any situation. Granted, the 
reliability and usefulness of self-assessment will de- 
pend, in part, on learners' self-confidence and skill 
in forming objective judgments However, with prac- 
tice, learners can improve these skills and become 
better lifelong learners. 

Learners can also become skilled in reviewing and 
evaluating their peers' performance, products, or 
knowledge. This makes peer evaluation an excellent 
technique for use with adults. A learner might re- 
quest another's help in evaluation— and later switch 
roles and become the evaluator. The student who 
acts as the peer evaluator can use a performance 
checklist or an answer key oi simp.y provide infor- 
mal feedback when rating the other's performance. 
After the initial peer evaluation, the students should 
meet to discuss the results and possible strategies 
for improvement. 

Peer evaluaticji can allow learners time to prac- 
tice skills and obtain objective feedback from peo- 
ple who are not involved in determining grades. Both 
|pj=»rners benefit from this process. One gams 
vatuable feedback about his or her achievement, the 
other sharpens his or her skills in identifying essen- 
tial cntena and thus reinforces :is or her own skills 
and knowledge. 

You will need to stress to leojners that they must 
be as objective, nonthreatening, and honest as 
possible in evaluation. Only in this way can their 
peers obtain useful, reliable information about their 
abilities. Note, too, that peer evaluation should usual- 
ly be voluntary. Some students may be extremely 
uncomfortable in either role, thus limiting the validi- 
ty or reliability of the evaluation. However, with prac- 
tice, many learners can acquire skill and ease with 
peer evaluations. 
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There may be times when neither you nor the 
learners are the most accessible or most expert eval- 
uator available. If a learner is completing an inde- 
pendent study in an area beyond your expertise, for 
example, an employer or worker with the needed 
expertise may need to be asked to evaluate the 
results. Likewise, in a cooperative education pro- 
gram in which learners are on the job part of the 
time, their on-the-job supervisors may need to take 
OP some of the responsibility for performance evalua- 
tions. The same might be true in an industry-based 
training program. 

In most situations, instructor evaluation should 
still be a substantial part of the evaluation system. 



In fact, the instructor's role as an evaluator is prob- 
ably the mcst vaned of all. With your knowledge of 
the subject area, you can help learners identify 
essential learning objectives, appropriate cnteria, 
and evaluation strategies. You can act as facilitator, 
helping learners to devise learning contracts and 
develop self-assessment and group decision-making 
skills. And of course, in conjunction with many col- 
laborative evaluation approaches, you will need to 
design, administer, and score written tests, design 
and evaluate performance tests, and submit evalua- 
tion reports to the institution in which you are 
teaching. 



Optional 
Activity 



You may wish to read one or both of the following supplementary references: 

• Knowles, The Modern Practice of Adult Education, examines the prin- 
ciples and practices involved in effective adult education. Section III, 
"Helping Adults Learn,'* pp. 222-247, discusses strategies to help 
learners diagnose their learning needs, formulate learning plans, and 
identify appropriate evaluation techniques. 

• Knox, Adut Development and Learning, is an in-depth and readable study 
of the many social, personal, and physical factors that affect adult 
development. Chapter 7, pp. 405-469, provides a detailed overview of 
the factors that influence and direct adult learning. 
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The following case study describes how Harv Dunfry, a new technical in- 
structor, developed evaluation plans for use with adult learners. Read the 
case study and critique in writing Mr. Dunfry's evaluation practices, both 
strengths and weaknesses. 



CASE STUDY 

Harv Dunfry hung up the phone and looked 
around the garage. "Not bad," he thougrit to himself. 
He had a three-bay garage now and steady 
customers who sent in referrals all the time. He'd 
come a p;etty long way from the high school kid 
who'd picked up extra money doing car repairs for 
neighbors. 

id now, one of his customers, who worked for 
the local school system, had just called. She*d asked 
him if he would be interested in teaching a beginning 
auto repair course in the night program for adults. 
He*d said, "Sure." 

Harv Dunfry, a teacher, eh? Well, school had been 
all right— except for the teachers who made you 
crazy with their rules and pop quizzes. But then, he 
didn't have to be that kind of teacher. He'd also had 
so.me good ones who really knew their stuff and whc 
made sure that you learned it. But at the same time, 
they had cared about who you were and what was 
important to you. That seemed to be more of what 
being a teacher was all about. 

In a few days there was supposed to be an orien- 
tation program for new instructors. Experienced in- 
structors were going to be teaching some techniques 
to use in the classroom so that you'd have something 
to go on when you first started. Harv hoped they'd 
have something useful to say. 

As It turned out, the last night of the orientation 
program had been particularly interesting. Someone 
had talked about working with adult students. She 
said they needed independence and the freedom to 
have a say in what they were going to learn and how 
they were going to be evaluated. 

It really had made a lot of sense. Wasn't that what 
his teachers in school had been doing when they let 
him work on some of his own projects and then 
helped him troubleshoot problems when he got 
stuck? Of course, he'd had to do regular studying 
and take tests, too, but that was usually fair. He 
might not have been able to work on some of the 
projects if he hadn't learned the material that the 
teachers reauired. 



All in all. It seemed like the best approach for his 
course might consist of some readings, followed by 
a few tests on the things that he knew for sure 
everyone would need to know. Then, by watching 
people in the shop, he could tell if they were learn- 
ing the hands-on skills that they needed. Other than 
that, everyone could decide about the additional 
things they wanted to learn and how they would be 
evaluated. That way, everyone could get the most 
out of the class. 

The first night of class Harv handed out an outline 
that described the learning objectives included in the 
course, the name of the book they would need to 
get, and a list of the six tests that would be given. 
Harv explained that the reason he was giving these 
tests was that they covered material that everyone 
would need to know if they were going to learn 
anything about auto mechanics. He also explained 
that what they did in the rest of the course was up 
to them. They would decide what they were going 
to learn and how they'd be tested, and then he'd help 
them in whatever way he could. 

He said that he'd heard about a process called 
group decision making that he'd like them to try out 
that evening. He explained how the process worked, 
and described all the steps involved. He then asked 
learners to form committees and decide on what else 
they wanted to learn and how they thought they 
should be evaluated. For example, did they want to 
take tests? Did they just want to talk to him about 
their work? Or were there other things they wanted 
to do? 

After the learners had formed committees, Han; 
left the room while everyone met. He didn't want to 
put any pressure on them about what they should 
do, and he thought it would be better if he just left 
everyone alone for a while. 

When they were done talking, everyone regrouped 
and Harv asked for their recommendations. It turned 
out that they all had pretty much agreed that the six 
tests he had on the outline were plenty. As for the 
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rest of the course, the general consensus was that 
he probably knew the most about what they should 
Isarn, since no one had worked much on cars before. 
After class he reflected, "So much for group deci- 
sion making— they all decide not to decide. Wei!, if 
that's what they want, I guess that's it." 

By the fourth week of the course, Harv wasn't sure 
where to go next. Everyone was still showing up, and 
they seemed to like the class. But they were all work- 
ing at different speeds. As a result, it was hard to 
decide when to start the next project on engine tim- 
ing, much less when to give the test on checking for 
worn-out belts. 

He'd thought after that first night that it would be 
pretty simple since everyone had agreed on the 
same thing. But, it wasn't— they all turned out to be 
so different. 

For instance, there was Lonnie, who didn't even 
know how to check his t>re pressure when he started 
the course (and he thought he was going to save 
hundreds of dollars in car repair bills by the time he 



finished the course!). He'd done all nght on the tests, 
but Harv wasn't sure he'd ever get air in his tires 
without blowing them up. Still, he sure seemed to 
be happy fooling around the engine, and he knew 
now to check his oil level now. So, maybe it wasn't 
a total loss. 

But then there were people like Janet and Steve, 
who were making rapid progress through the hands- 
on activities. They evidentally had done some work 
on cars before. They kept talking about exhaust 
systems and rebuilding carburetors. He didn't want 
to hold them back just because other people weren't 
as far along. 

On the other hand, Steve hadn't passed any of 
the tests so far. For that matter, he hadn't even gone 
so far as to write his name on the test. Maybe he 
just was too embarrassed to put his name on a test 
he hadn't studied for. At first, Harv thought maybe 
Steve had been sick and hadn't had a chance to 
study, so he asked him if he wanted to take tne tests 
again later. But Stev3 said no, he'd just live with it. 
We^ll, okay. He didn*t want to push anybody. 
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Compare your written critique of the instructor's performance with the model 
critique given below. Your response need not exactly duplicate the model 
response; however, you should have covered the same major points. 



MODEL CRITIQUE 

Harv Dunfry's approach to teaching and evaluation 
was based on the very best of intentions. Drawing 
on some good general advice given at the instruc- 
tor's orientation session and his memory of good 
teachers he'd had in hi^h school, Harv set out to 
teach a class in which adult learners had choices 
aDout what they would learn and how they would be 
evaluated. Clearly, he had a lot to offer the learners 
who returned to the course week after week. 

He should be commended for relying on his 
knowledge and experience when he provided struc- 
ture for the course by identifying the performance 
objectives and evaluation devices essential for an 
introductory course. 

Harv's choice in using a collaborative approach 
was a good one, and he made a very sensible move 
when he explained to learners all of the steps involved 
in the group decision-making process. However, 
Harv*s silence on the subject of grades left a number 
of important questions unanswered. Were the 
planned tests to bo graded? What other learning ac- 
tivities would be graded, and ho/v? If so, what 
assessment results would contribute to a final 
grade? Would grades be submitted to the institution? 
Did the school require that learners be graded? 
Harv*s students needed to have this information in 
order to make more informed decisions about their 
activities and appropriate evaluation strategies. 

In addition, even though he was acting out of 
respect for the learneis' right to make independent 
decisions, Harv's decision to leave the classroom 
during the committee sessions .vas unwise. He 
needed to be present in order to provide informa- 
tion and guidance during the process. As it was, 
beginning learners simply didn't have enough infor- 
mation to work with when making decisions. Hence, 
thoir choice was "not to decide.'* 



Harv was thrown off track by the class's general 
agreement to do what he thought best. It was possi- 
ble, since this was an introductory course, that he 
knew the most about specific learning options. 
However, he could have presented learners with a 
variety of learning experiences and evaluation ac- 
tivities lO choose from, rather than simply telling 
them to decide for themselves. 

It is also possible that, given learners* range of 
skills and experience, Harv and the class would have 
been better off using individual learning contracts. 

Harv*s concern about holding back some students 
because others were slower could have been elimi- 
nated if he had conducted some preassessment ac- 
tivities to obtain basic information about learners' 
skill and knowledge levels. It turned out that his 
students had a wide range of skill levels, which could 
have been accommodated earlier if he had known 
about them. As it was, Harv reiied on observation 
and informal discussion to obtain this information. 
These are indeed good tools for assessment if used 
systematically, but they need to be supplemented 
with other strategies tPat can provide specific infor- 
mation at an early poini in ihe course. 

Perhaps Harv would have discovered, through 
©reassessment, that Steven had some advanced 
skills in auto mechanics but lacked other skills, such 
as the ability to read or write well. With this knowl- 
edge, he could have done two important things. 
(1) work with Steven to identify more advanced proj- 
ects to work on independently and (2) evaluate 
Steven's knowledge through oral, rather than wnt- 
ten, tests. 

Harv was correct in explaining why he had 
developed the tests that he planned to use. but he 
should have also explained how the test results 
could help them, as well as how they affected overall 
grades. 
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It also seems that, even though the use of some 
written tests was appropnate for assessing learners* 
knowledge, Harv needed to go further in his plans 
and provide for evaluating learners' hands-on bKills 
through observation. He needed to develop some 
performance tests to accurately evaluate learners' 
skills. Learners could also have used these tests for 
practice and self-assessment. Maybe, in that way, 
Lonnie could even learn to fill his tires without an 
explosion. 

In addition, a group consensus about taking the 
tests that Harv had planned did not mean that every- 
one in the class had to take the same test at the 



same time. There was no apparent need for this. And 
if he had arranged for learners to take tests when 
they were ready, there would have been less confu- 
sion about when to administer the next test and 
when to move on to the next learning expenence. 

For the most part, Harv Dunf.7's efforts in work- 
ing with adults to develop evaluation strategies were 
pretty well founded, but they needed to be further 
thought out and developed. 



Level of Performance: Your written cr»t:que of the case study should have covered the same major points 
as the model critique. If you missed some points or have questions about any additional points you made, 
review the material in the information sheet, Evaluating Adults. What's Appropriate?, pp. 50-57, or check 
with your resource person if necessary. 
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Learning Experience IV 



FINAL EXPERIENCE 




NOTES 
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^ TEACHER PERFORMANCE ASSESSMENT FORM 

Evaluate the Performance of Adults {N-6) 



Name 

Directions: Indicate the Ibvel of the teacher's accomplishment by placing 

an X in the appropriate box under the LEVF' OF PERFORMANCE heading. Date 

If, because of special circumstances, a perforri.ance component was not ap- 

plicable, or impossible to execute, place an X in the N/A box. Resource Person 



LEVEL OF PERFORMANCE 



• 



in planning for evaluation, the instructor: 

1. identified and accommodated the following: 

a. institutional requirements 

b. occupational requirements 

c. individual needs, objectives, and skill and knowledge 
levels 

d. learners' special needs 

In using collaborative approaches to developing evaluation 
strategies, the instructor: 

2. worked with learners, to the extent possible, to identify: 

a. learning objectives 

b. learning experiences 

c. evidence of accomplishment 

d. evaluation criteria 

e. evaluation strategies 

f. timing of evaluation 

If the group decision-making process was used, the 
instructor: 

3. introduced the process and the procedures to be followed 

4. accepted and listed aii learner recommendations for 
discussion 

5. guided and contributed to the discussion of the recommen- 
dations as needed 

6. ensured that consensus was reached 
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If learning contracts were used, the instructor: 

7. provided learners with adequate structure and directions 
for developing learning contracts 

8. worked with individual learners to design appropriate 
contracts 

9. provided a copy of the signed contract for both the learner 
and the instructor 

10. reviewed contracts with learners on a regular basis and 
made any necessary adjustments 

If grading contracts were used, the instructor: 

1 1 . identified the relative weight ranges lOr evaluation activities 
or the amount of work to be performed for each grade 

12. provided copies of the signed contract for the instructor 
and the learner 

in selecting the appropriate evaluators, the instructor: 

1 3. ensured that a variety of evaluators were involved through 
such strategies as: 

a. self-evaluation 

b. peer evaluation 

c. instructor-based evaluation 

d. expert worker/supervisor evaluation 

In developing each written test, the instructor: 

14. identified appropriate types of test items based on the per- 
formance objectives 

15. developed each test item according to the guidelines for 
constructing that type of item 

16. developed a test (hat was valid, reliable, and usable 

17. provided clear, readable copies for everyone 

18. provided full, clear, and simple directions for the whole test 
and for each type of test item 

19. grouped tho same kinds of test items together 

20. included no more than three different types of test items 

21. developed clear and unambiguous test items, with the 
guessing factor minimized 

22. developed an appropriate scoring key 
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In developing ench performance test, the Instructor: i — i 

23. stated the performance objective fully and clearly .... I I 

24. provided clear directions concerning the testing situation EH 

25. laid out the checklist/rating scale so that it is clear and easy i — i 
to use I I 

26. included all essential criteria and only essential criteria CZJ 

27. stated all items clearly EH 

28. arranged all items in a logical sequence EH 

29. ensured that the test was: « — i 

a. appropriate for the type of performance involved ... 1 J 

b. reasonable in length O 

In administering each test, the instructor: i — i 

30. provided a comfortable testing environment I I 

31. explained the purpose of the test prior to administering it EH 

32. allowed sufficient time for the test to be completed by all i — i 
or most learners I I 

After administering each test, the instructor: 

33. reviewed the results and identified the following: 

a. learners' skill and knowledge areas that need to be | — i 
strengthened i 1 

b. any necessary changes in instructional plans EH 

In determinmg learners' grades, the instrucior: 

34. informed learners of the basis and system for determin- i — j 
ing their grades I I 

35. maintained complete ana accurate record:; of learner i — j 
performance I I 

36. assigned grades that were consistent with the institution's » — i 
grading system u 
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Level of Performance: All items must receive N/A, GOCD, or EXCELLENT responses. If any item receives 
a NONE, POOR, or FAIR response, Ine instructor and resource person should meet to determine what 
additioi al activities th instructor needs to complete m order to reach competency in the weak ared(s). 
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NOTES 
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ABOUT USING THE NATIONAL CENTER'S 
PBTE MODULES 



OrganiZiiion 

Each module is designed to help you gain competency in a 
particular skill area considered important to teaching sue* 
cess. A module is made up of a series of teaming experi* 
ences, some providing background information, some pro- 
viding practice experiences, and others combining these 
two functions. Completing these experiences shOLid en- 
able you to achieve the terminal objective in the final 
learning experience. The final experience in each module 
always requires you to demonstrate the skill in an actual 
teaching situation when you are an intem, a student teach- 
er, an inservice teacher, or occupational trainer. 

Procedures 

Modules are designed to allow you to individualize your 
teacher education program. You need to take only those 
modules covering skills that you do not already possess. 
Similarly, you need not complete any learning experience 
within a module if you already have the skill neede^ to 
complete it. Therefore, before taking any modu' you 
should carefully review (1) the introduction, (2) the objec- 
tives listed on p. 4, (3) the overviews preceding each team- 
ing experience, and (4) the final experience. After compar- 
ing your present needs and competencies with the. informa- 
tion you have read in these sectir is, you should be ready to 
make one of the following decisions: 

• That you do not have the competencies indicated and 
should complete the entire module 

• That you are competent in one or more of the enabling 
objectives leading to the final learning experience and, 
thus, can omit those ieaming experiences 

^ "Hiat you are already competent in this area and are 
3ady to complete the final learning experience in 
order to "test out" 

• That the module Is inappropriate to your needs at this 
time 

When you are ready to complete the final learning expen- 
ence and have access to an actual teaching situation, 
make the necessary arrangements with your resource per- 
son. If you do not complete the final experience success- 
fully, meet with your resource person and arrange to (1) 
repeat the experience or (2) complete (or review) previous 
sections of the module or other related activities suggested 
by your resource person before attempting to repeat the 
final experience. 

Options for recycling are also available in each of the 
learning experiences preceding the final expenence. Any 
time you do not meet the minimum level of performance 
required to meet an objective, you and your resource per- 
son may meet to select activities to help you reach compe- 
tency. This could involve (1 ) completing parts of the module 
previously skipped, (2) repeating activities, (3) reading sup- 
ple mentary resources or completing additional activities 
suggested by the resource person, (4) designing your own 
learning experience, or (5) completing some other activity 
suggested by you or your resource person. 



Terminology 

Actual Teaching Situation: A situation in which you are 
actually worthing with and responsible for teaching sec- 
ondary or postsecondary vocational students or other oc- 
cupational trainees. An Intem, a student teacher, an in- 
service teacher, or other occupational trainer would be 
functioning in an actual teaching situation. If you do not 
ha d access to an actual teaching situation when you are 
taking the module, you can complete the module to the 
final learning experience. Vou would then complete the 
final leaming expenence later (i.e., when you have access 
to an actual teaching situation). 
Alternate Activity or Feedback: An item thai may substi-. 
tute for required items that» due to special circumstances, 
you are unable to complete. 

Occupational Specialty: A specific area of preparation 
within a vocational sen/ice area (e.g., the service area 
Trade and Industrial Education includes occupational spe- 
cialties such as automobile mechanics, welding, and elec- 
tricity. 

Optional Activity or Feedback: An Item that is not re- 
quired but that is designed to supplement and ennch the 
required items in a learning experience. 
Resource Person: The person in charge of your educa- 
tional program (e.g., the professor, instructor, administrator, 
instructional supervisor, cooperatinc supervising/class- 
room teacher, or training supervisor v\ o is guiding you In 
completing this module). 

Student: The person who is receiving occupational in- 
struction in a secondaiy, postsecondary, or other training 
program. 

Vocational Sen/ice Area: A major vocational field, agri- 
cultural education, business and office education, mart^et- 
mg and distributive education, health occupations educa- 
tion, home economics education, industrial arts education, 
technical education, or trade and industrial education. 
You or the Teacher/Instructor: The person who is com- 
pleting the module. 

Levels of Performance for Final Assessment 
N/A: The criterion was not met because it was not appli- 
cable to the situation. 

None: No attempt was made to meet the criterion, al- 
though it was relevant. 

Poor: The teacher is unable to perform this skill or has or.ly 
very limited ability to perform it. 
Fair: The teacher is unable to perform this skill in an ac- 
ceptable manner but has some ability to perform it. 
Good: The teacher is able to perform this sk:ll in an effec- 
tive manner. 

Excellent : The teacher is able to perfonn this skHI in a very 
effective manner. 
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Titles of the National Center's Performance-Based Teacher Education Modules 



Category A: Program Planning, Development, and Evaluation 

A-1 Prepare lo» i Communrty Survey 

A-2 Conduct a Community Survey 

A«3 Report the Findings ct a Community Survey 

A-4 Organize an Occupational Advisory Committee 

A-5 Maintain an Occupational Advisory Committee 

A-6 Oevetop Program Goals and Objectives 

A-7 Conduct an Occupational Analysis 

A-8 Develop a Course of Study 

A-9 Develop Long-Range Program Plans 

A-10 Conduct a Student Follow-Up Study 

M 1 Evaluate Your Vocational Program 

Category B: Instructional Planning 

6*1 Determine Needs and Interests of Students 

6*2 Develop Student Performance Objectives 

Develop a Unit of Instruction 

Develop a Lesson Plan 

B-5 Select Student Lnstructional Matenals 

B-6 Prepare Teache.'-Made Instructional Matenals 

Category C: Instructional Execution 

C-1 Direct Field Tnps 

C>2 Conduct Group Discussions, Panel Discussions, and Symposiums 

C-3 Employ Brainstorming. Buzz Group, and Question Box Techniques 

C-4 r.rect Students in instructing Other Students 

C*5 Employ Simulation Techniques 

C-G Guide Student Study 

C-7 Direct Student Laboratory Expenence 

C-8 D''ect Students in Applying Problem-Solving Techniques 

C-9 Employ the Project Method 

C-10 Introduce a Lesson 

C^^^ Summanze a Lesson 

C'12 Employ Oral Questioning Techniques 

C-1 3 Emplo'/ Reinforcement Techniques 

C*14 Provide instruction for Slower and More Capable Learners 

C-1 5 Present an Illustrated Talk 

C-1 6 Demonstrate a Manipulative Skill 

C'17 Demonstrate a Concept or Principle 

C-1 8 individualize Instruction 

C-1 9 Employ the Team Teaching Approach 

C-20 Use Subject Matter Experts to Present Information 

C-21 Propane Bulletin Boards and Exhibits 

C-22 Present Information with Models, Real Objects, and Flannel Boards 

C-23 Present Information with Overhead and Opaque Matenals 

C-24 Present Information with Filmstrips and Slides 

C-25 Present Information with Films 

C-26 Present tnfon *ation with Audio Recordings 

C-27 Present Information wi h Televised and Videotaped MatertaSs 

C-28 Employ Programmed Instruction 

C-29 Present Information with the Chalkboard and Flip Chart 

Category D: Instructional Evaluation 

D>1 Establish Student Performance Criteria 

D-2 Assess Student Performance Knowledge 

D-3 Assess Student Performance* Attitudes 

0-4 Assess Student Performance Skills 

D'5 Determine Student Grades 

D-€ Evaluate Your Instructional Effectiveness 

Category E: Instructional Management 

E'l Prefect InJtrucbonal Resource Needs 

E-2 Manage Your Budgeting and Reporting Responsibilities 

E*3 Arrange for Improvement of Your Vocational Facilities 

E-4 Maintain a Filing System 

E'5 -rovKfe for Student Safety 

E-6 Provide for the First Aid Needs of Students 

E*7 Assist Students in Developing Self*Disciotine 

E*8 Organize the Vocational L::i>oratory 

E-9 Manage the Vocational Laboratory 

E'\0 Combat Problems of Student Chemical Use 

Category F: Guidance 

F . Gather Student Data Uang Format Data-Collection Techniques 

F.2 Gather Student Data Through Personal Contacts 

F-3 Use Conferences to Help Meet Student Needs 

F-4 Provide lnforr»)ation on Educational and Career Opportunities 

F-5 Assist Students in Applying for Employment or Further Education 



Category G: School-Community Relations 

G«1 Develop a School-Community Relations Plan for Your Vocational Program 

G-2 Grve Presentations to Promote Your Vocational Program 

G-3 Develop Brochu'^^T to Promote Your Vocational Program 

G-4 Prepare Displays to Promote Your Vocational Program 

G-5 Prepare News Rel«^ases ar>d Articles Concerning Your Vocational Program 

G-6 Arrange for Telev.sion and Radio Presentations 

Concerning Yoi'r Vocational Program 

G-7 Conduct an Open House 

G'8 Work with Members of the Community 

G-9 Work with State and Local Educators 

G-10 Obtain Feedback abo'j( Your Vocational Program 
Category H: Vocational 'Student Organization 

H-1 Develop a Personal Ph losophy Concerning 

Vocational Student Organizations 

H*2 Establish a Vocational Student Organization 

H-3 Prepare Vocational Student Organization Members for Leadership Roles 

H-4 Assist Vocational Student Organization Members in Developing and 

Financing a Yearly Program of Activities 

H-5 Supervise Activities of tho Vocational Student Organization 

H-6 Guide Participation in Vocational Student Organization Contests 
Category I: Professional Role and Development 

M Keep Up-to-date Professionally 

1*2 Serve Your Teaching Profession 

t-3 Develop an Active Personal Philosophy of Education 

1-4 Serve the Schcol and Community 

1*5 Obtain a bunaMe Teaching Position 

t-6 Provide Laboratory Expenences for Prospective Teachers 

1*7 Plan the Student Teaching Expenence 

1-8 Supervise Student Teachers 
Category J: Coordination of Cooperative Education 

J'l Establish Guidelines for Your Cooperative Vocational Program 

J-2 Maiiage the Attendance, Transfers, and Terminations of Co-op Students 

J*3 Enroll Students in Your Co-op Program 

J-4 Secure Training Stations for Your Co-op Program 

J*5 Place Co-<^ Students on the Job 

J'6 Develop the Training Ability of On-theJob Instructors 

J*7 Coordinate Orvthe-Job Instruction 

J*8 Evaluate Co-op Studcits' On'theJob Performance 

J-9 Prepare for Students' helated Instruction 

J- 10 Supervise an Employer* Employee Appreciation Event 
Category K: Implementing Competency-Based Education (CBE) 

K-1 Prepare Yourself for CBE 

K-2 Organize the Content for a CBE Program 

K-3 Organize Your Class and Lab to Install CBE 

K-4 Provide Instructional Matenals for CBE 

K-5 Manage the Daily Routines of Your CBE Program 

K-€ Guide Your Students Through the CBE Program 
Category L: Serving Students with Special/Exceptional Needs 

L 1 Prepare Yourself to S^rve Exceptional Students 

L*2 Identify and Diagnose Exceptional Students 

L*3 Plan Instruction for Exceptional Students 

L-4 Provide Appropriate Instructional Matenals for Exceptional Students 

L*5 Mooify the Learning Environment for Exceptional Students 

L*6 Promote Peer Acceptance of Exceptional Students 

L-7 ise Instructional Techniques to Meet the Needs of Exceptionat Students 

L*8 Improve Your Communication Skills 

L*9 Assess the Progress of Exceptional Students 

L*10 Counsel Exceptional Students with Person af'Soci at Problems 

L'll Assist Exceptional Students in Developing Career Planning Ckills 

L*12 Prepare Exceptional Students for Employability 

L-^"* Promote Your Vocational Program with Exceptional Students 

Category M: Assisting Students In Improving Their Basic Skills 

M-1 Assist Students w Achieving Basic Reading Skills 

M-2 Assist Students n Developmg Technical Reading Skills 

M-3 Assist Students in Improving Their Wnting Skills 

M-4 Assist Students in Improving Their Oral Communication Skills 

M*5 A^ist Students in Improving Their Math Skills 

M-6 Assist Students m Improving Their Survival Skills 

Category N: Teaching Adults 

N-1 Prepare 'o Work with Adult Learners 

N-2 Market an Adult Education Program 

N*3 Determine Individual Training Needs 

N-4 Plan Instruction for Adults 

N*5 Manage the Adult Instructional Process 

N-6 Evaluate the Performance of Adults 
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